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Overview 



Although much is known about how to help welfare recipients find jobs, little is known about 
how to help them and other low-wage workers keep jobs or advance in the labor market. This 
report assesses the implementation and two-year follow-up effects of a program in Texas that 
aimed to promote job placement, employment retention, and advancement among applicants 
and recipients in the Temporary Assistance for Needy Families (TANF) program. The Texas 
program is part of the Employment Retention and Advancement (ERA) project, which is testing 
15 such programs across the country. The ERA project is being conducted by MDRC, under 
contract to the U.S. Department of Health and Human Services, with additional funding from 
the U.S. Department of Labor. 

To encourage employment retention and advancement among working TANF leavers, the 
Texas ERA program provided job search assistance, pre- and postemployment case manage- 
ment, and a monthly stipend of $200. The program was evaluated in three sites — Corpus 
Christi, Fort Worth, and Houston — starting in 2000. The ERA evaluation uses a random as- 
signment research design: Through a lottery-like process, eligible individuals were assigned 
either to a program group, whose members participated in the ERA program, or to a control 
group, whose members participated in Texas’s standard welfare-to-work program (called 
“Choices”). The control group’s outcomes tell what would have happened in the absence of the 
ERA program, providing benchmarks against which to compare the program group. 



Key Findings 

• The ERA program was well implemented in Corpus Christi but experienced some 
operational difficulties in Fort Worth and Houston. Across the sites, the control group 
participated in a relatively strong welfare-to-work program. ERA and Choices ended up 
being quite similar during the preemployment phase but had larger treatment differences 
during the postemployment phase, primarily due to the stipend. A significant effort was 
needed to market the stipend, and program staff increasingly made a good- faith effort to do 
so over time. In the end, however, this effort may have been insufficient, given that people 
needed to use up a four-month TANF earnings disregard before becoming eligible for the 
stipend. Among all those randomly assigned to ERA, stipend receipt rates were about 30 
percent in Corpus Christi and 20 percent in the other sites; among those who found jobs 
and received the entire earned income disregard, estimated stipend receipt rates were about 
55 percent in Corpus Christi and 40 percent in the other sites. 

• The Texas ERA program did not produce consistent or large effects on employment 
and retention outcomes during the first two years of the study period. In Corpus 
Christi, there were a few modest impacts on employment and retention that were concen- 
trated among those who entered the program during the early phases of the study period; 
the extra income from the stipend was enough to generate a statistically significant effect 
on income. In Fort Worth, the program’s impacts were on initial employment rather than 
on employment retention. There were few impacts in Houston. 

MDRC will continue to track employment outcomes for the study’s participants, so these results are 
not the final word on the Texas ERA program. Yet die results do rei nf orce the view that promoting 
employment retention and advancement among welfare recipients presents challenging implementa- 
tion issues. 
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About the Employment Retention and 
Advancement Project 



The federal welfare overhaul of 1996 ushered in myriad policy changes aimed at getting 
low-income parents off public assistance and into employment. These changes — especially 
cash welfare’s transformation from an entitlement into a time-limited benefit contingent on 
work participation — have intensified the need to help low-income families become economi- 
cally self-sufficient and remain so in the long term. Although a fair amount is known about how 
to help welfare recipients prepare for and find jobs in the first place, the Employment Retention 
and Advancement (ERA) project is the most comprehensive effort thus far to discover which 
approaches help welfare recipients and other low-income people stay steadily employed and 
advance in their jobs. 

Launched in 1999 and slated to end in 2008, the ERA project encompasses more than a 
dozen demonstration programs and uses a rigorous research design to analyze the programs’ 
implementation and impacts on research sample members, who were randomly assigned to the 
study groups. With technical assistance from MDRC and The Lewin Group, the study was con- 
ceived and funded by the Administration for Children and Families in the U.S. Department of 
Health and Human Services; supplemental support comes from the U.S. Department of Labor. 
Most of the ERA programs were designed specifically for the purposes of evaluation, in some 
cases building on prior initiatives. Because the programs’ aims and target populations vary, so 
do their services: 

• Advancement programs focus on helping low-income workers move into 
better jobs by offering such services as career counseling and education and 
training. 

• Placement and retention programs aim to help participants find and hold 
jobs and are aimed mostly at “hard-to-employ” people, such as welfare re- 
cipients who have disabilities or substance abuse problems. 

• Mixed-goals programs focus on job placement, retention, and advancement, 
in that order, and are targeted primarily to welfare recipients who are search- 
ing for jobs. 

The ERA project’s evaluation component investigates the following aspects of each 
program: 

• Implementation. What services does the program provide? How are those 
services delivered? Who receives them? How are problems addressed? 
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• Impacts. To what extent does the program improve employment rates, job 
retention, advancement, and other key outcomes? How does it affect enrol- 
lees’ children? Looking across programs, which approaches are most effec- 
tive, and for whom? 

A total of 15 ERA programs are being implemented in eight states: 

• California: Los Angeles County and Riverside County 

• Illinois: Cook County (Chicago) and St. Clair County (East St. Louis) 

• Minnesota: Hennepin County (Minneapolis) 

• New York: New York City 

• Ohio: Cleveland 

• Oregon: Eugene, Medford, Portland, and Salem 

• South Carolina: Pee Dee Region (six counties in the northeast comer of the 
state) 

• Texas: Corpus Christi, Fort Worth, and Houston 

The evaluation draws on administrative and fiscal records, surveys of participants, and 
field visits to the sites. 
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Executive Summary 



This report presents an assessment of the implementation and the two-year impacts of a 
program in Texas that aimed to promote job placement, employment retention, and advancement 
among applicants and recipients to the Temporary Assistance for Needy Families (TANF) pro- 
gram. The program in Texas is part of the Employment Retention and Advancement (ERA) pro- 
ject, which is testing 15 programs across the country. 1 The ERA project was conceived and 
funded by the Administration for Children and Families in the U.S. Department of Health and 
Human Services (HHS) and is also supported by the U.S. Department of Labor. The project is 
being conducted by MDRC, a nonprofit, nonpartisan research organization, under contract to HHS. 

The ERA program in Texas was evaluated in three cities (sites) — Corpus Christi, Fort 
Worth, and Houston — beginning in the fall of 2000. Most of the employment outcomes pre- 
sented in this report cover the first two years after individuals entered the program. The results 
include the program’s effects on employment levels and stability, earnings, and advancement in 
the labor market. These results are important but are not the final word on the program, as 
MDRC will track employment outcomes for the study’s participants for a total of three years. 



The ERA Project 

Although much is known about how to help welfare recipients find jobs, little is known 
about how to help them and other low-wage workers keep jobs or advance in the labor market. 
Previously studied postemployment programs were not found to improve participants’ out- 
comes. The ERA project was designed to build on past efforts and identify and test innovative 
programs designed to promote employment stability and wage progression among low-income 
groups, including welfare recipients. From 2000 to 2003, a total of 15 experiments were imple- 
mented in eight states, including Texas. 

The design of the evaluation is similar in most of the project’s sites. Individuals who 
meet the ERA eligibility criteria, which vary by site, are assigned at random to a program 
group, called the ERA group, or to a control group. Members of the ERA group are recruited 
for (and, in some sites, are required to participate in) the ERA program, while those in the con- 
trol group are not eligible for ERA services but can access other services and supports available 
in the community. MDRC is tracking both research groups over time. The random assignment 
process ensured that the two groups were comparable when they entered the study; thus, any 



'Although the Texas program operated in various cities that are called “sites” in this report, Texas counts 
as a single ERA site. 
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differences between them that emerge over time — for example, in employment rates or aver- 
age earnings — can be confidently attributed to the ERA program. 2 



The Texas ERA Program 

The Texas ERA program was designed to promote job placement, retention, and career 
advancement for TANF applicants and recipients. The program provided both pre- and postem- 
ployment services and targeted a population applying for or receiving cash assistance, most of 
whom were not working when they entered the program. The Texas ERA program included job 
placement, employment stabilization, and advancement services — along with a monthly sti- 
pend of $200 for working TANF leavers, to encourage employment retention and advancement. 

The Texas ERA program was developed by the Texas Department of Human Services 
(DHS), in coordination with the Texas Workforce Commission (TWC). DHS was primarily 
responsible for determining eligibility and overseeing the TANF cash assistance program, while 
TWC managed TANF employment services. TWC oversaw employment services for the ERA 
program as well as for Choices — the standard program in the state that provides employment- 
related services to TANF recipients. As designed, the Texas ERA program was to include the 
following services: 

• Preemployment job search and team-based case management services. 

For most participants, the ERA program initially provided job search, job 
readiness, and case management services. Because postemployment case 
management services had little effect on employment outcomes in past reten- 
tion and advancement studies, Texas strengthened these services by having 
them begin at the preemployment stage and using a team-based approach. 

The goal potentially was to involve partners from multiple agencies (includ- 
ing DHS, local workforce staff, and organizations working to prevent sub- 
stance abuse or domestic violence) that had expertise in addressing specific 
employment-related barriers. Key case management services included em- 
ployment assessment, goal setting and career planning, support services, 
resolution of employment barriers, and job search assistance. 

• Stipend for welfare leavers. The ERA program provided a monthly stipend 
of $200 to participants who left TANF, were employed for a minimum of 30 

2 For more information on the ERA project, see Bloom, Anderson, Wavelet, Gardiner, and Fishman, New 
Strategies to Promote Stable Employment and Career Progression: An Introduction to the Employment Reten- 
tion and Advancement Project (U.S. Department of Flealth and Fluman Services, 2002). For early results from 
four sites, including Texas, see Bloom, Flendra, Martinson, and Scrivener, The Employment Retention and 
Advancement Project: Early Results from Four Sites (U.S. Department of Flealth and Fluman Services, 2005). 
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hours per week, and participated in a postemployment “advancement” activ- 
ity. The stipend was also available to those who combined 15 hours of work 
per week with an education and training activity that lasted 15 hours per 
week. Before receiving the stipend, participants had to first exhaust the four- 
month earned income disregard, which allowed all welfare recipients who 
left the rolls to continue to receive most of their full welfare grant for four 
month s . 3 There was a lifetime limit of 12 stipends (which did not have to be 
used in consecutive months). The stipend was included in the ERA model 
based on other studies that found similar earnings supplements to be effective 
in encouraging job retention and increasing earnings. 4 

• Intensive postemployment services. The ERA program provided a com- 
prehensive set of postemployment services, which could include assistance 
with job-related problems and support services, monitoring job performance 
and issues through regular site visits to employers, rapid reemployment assis- 
tance for participants who lost jobs, and support in meeting the requirements 
of the stipend. ERA postemployment services could continue for as long as 
an individual was eligible for the stipend. 



The Design of the Evaluation 

Immediately following an eligibility or recertification interview for TANF (but, in the 
case of applicants, before they were approved for cash assistance), individuals were randomly as- 
signed either to the ERA program or to the Choices program. Those who were assigned to ERA 
were introduced to the program and were then required to attend an orientation on the TANF pro- 
gram before being approved for cash assistance. Once these individuals were approved for TANF, 
they were then engaged in ERA program services. Individuals who were not approved for TANF 
were not eligible for the ERA program. Random assignment began in October 2000. 

Once assigned to the ERA program, individuals completed an assessment; then they 
generally participated in job search and, if they did not find a job, community service (a volun- 
teer position in a nonprofit or public organization); and then they received case management 
services. Once employed, participants received postemployment services, and — after leaving 



^Depending on participants’ earnings, the amount of the residual TANF grant combined with the earned 
income disregard was roughly equivalent to or somewhat less than the $200 stipend. Individuals could receive 
the earned income disregard once in a 12-month period (but the four months did not have to be used consecu- 
tively within this period). 

4 See Gordon Berlin, Encouraging Work, Reducing Poverty’: The Impact of Work Incentive Programs 
(New York: MDRC, 2000). 
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TANF and after receiving the earned income disregard — they became eligible for the monthly 
stipend. Participation in the ERA program was mandatory for most individuals, meaning that 
they faced a reduction or termination of TANF benefits for noncompliance with program rules. 

Individuals in the control group were assigned to participate in Choices, the state’s 
standard welfare-to-work program, which also provided pre- and postemployment services to 
TANF recipients. In terms of preemployment services, individuals who were assigned to 
Choices — like their counterparts in ERA — also completed an assessment, participated in job 
search, and did community service if they did not find employment. Choices also provided case 
management and support services, but its staff generally did not use the team-based approach 
that was developed for ERA and did not typically engage in longer-term career planning. Again 
like ERA, participation in Choices was mandatory for most TANF recipients. 

There were greater differences between ERA and Choices in the nature of postem- 
ployment services. Employed individuals in Choices were not eligible for the $200 stipend, al- 
though they could receive the earned disregard for up to four months (at the same level as under 
the ERA program). Under Choices, postemployment case management services generally lasted 
only for the duration of the earnings disregard (whereas services continued for up to an addi- 
tional 12 months for workers receiving the stipend under ERA), and they were less intensive 
than the postemployment services provided through the ERA program. 



Key Findings on Program Implementation 

This section summarizes the report’s findings on how the Texas ERA program was im- 
plemented and on sample members’ participation in the program and other employment-related 
services. The findings are based on field research, a time study of ERA staff, automated pro- 
gram tracking data on stipend receipt, and a survey administered to a subset of sample members 
about 12 months after they entered the study. Key implementation findings follow. 

• While ERA was intended to be distinct from Choices in terms of both 
pre- and postemployment services, most of the key differences were in 
the nature of the postemployment services, particularly the stipend. 

In all the Texas sites, the job search and case management services, participation man- 
date, and support services provided under Choices resulted in the control group’s receiving a 
relatively strong set of preemployment services that were similar to those provided by ERA. 
The Choices program also experienced some improvements over the course of the study period. 
Although “team-based” case management (involving partners from multiple agencies) was an 
important concept in the development of the ERA program — and one that was intended to dis- 
tinguish it from the Choices program — in the end the preemployment case management ser- 
vices provided by ERA were similar to those provided by Choices. 
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For those who found jobs, ERA’s retention and advancement services were strongest 
and most distinct from Choices after the four-month earnings disregard period — once indi- 
viduals were receiving a stipend. At this point, participants began working most closely with the 
postemployment staff, typically meeting with them at least once a month. Program staff in Cor- 
pus Christi and Fort Worth often conducted this monthly meeting at the worksite of participants 
who were receiving a stipend — although employer site meetings did not start in Fort Worth 
until later in the study period. 

• Overall, Corpus Christi achieved the highest stipend receipt rate, with 
30 percent of the program group receiving at least one payment (com- 
pared with 20 percent in the other sites). All the sites in Texas found that 
significant effort had to be put into marketing the stipend. 

Some individuals in the program group did not qualify for the ERA stipend because 
they did not find jobs or they did not work longer than the four-month earned income disregard 
period. Individuals who received the entire earned income disregard generally became eligible 
for the stipend — although they might not have received it in a given month if they did not 
work enough hours, submit the required documentation, or attend the required activity. 

Figure ES.l shows the steps involved in receiving a stipend in Corpus Christi. Out of 
every 100 people randomly assigned to ERA, 84 met the first criterion for receiving a stipend: 
They were determined eligible for the TANF program and received benefits. Of those 84 peo- 
ple, 70 became employed at some point during the follow-up period, but only an estimated 55 
of them appear to have worked enough to receive the entire four-month earnings disregard. 5 Of 
those 55 who worked at least four months, 30 people (55 percent) received a stipend. In Fort 
Worth and Houston, approximately 40 percent of those who worked longer than four months 
received a stipend (not shown in the figure). 

All the sites in Texas developed a solid marketing effort for the ERA stipend, particu- 
larly over time, with Corpus Christi moving most quickly to develop marketing materials and 



5 Because individuals could receive the earned income disregard only once in a 12-month period (but the 
disregard did not have to be used in consecutive months), the population that was eligible for the stipend was 
estimated — using unemployment insurance (UI) data — as those employed individuals who worked in two or 
more quarters within a year and who earned over $2,400 in these two quarters (the equivalent of working at $7 
per hour for 20 hours per week). This was done to eliminate those who worked very little over the two quarters 
and would not have received the disregard for the full period. Because the earnings threshold that is used to 
determine who would be eligible for a stipend is an estimate, a sensitivity analysis was conducted to determine 
how the results would change using different earnings thresholds. Overall, no large differences were found in 
the number who were eligible for the stipend. 
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The Employment Retention and Advancement Project 
Figure ES.l 

Estimated Eligibility for and Use of the ERA Stipend Among 100 TANF Applicants and 

Recipients in Corpus Christi a 




SOURCES: MDRC calculations from UI administrative records from the State of Texas and ERA program 
tracking data. 

NOTES: “Among all ERA group members randomly assigned from October 2000 through January 2003. 

b The proportion of individuals who worked for four months and thereby completed the earnings disregard 
period was estimated using UI records. Individuals were determined to have worked for four months if they 
were employed for two consecutive quarters within a year and had total earnings of more than $2,400 during this 
period (this is the equivalent of working 20 hours per week for four months at $7 per hour). 
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strategies. However, with only about half of those who were eligible receiving the stipend, there 
was still clearly room for improvement. The reasons for not using the stipend among those who 
were eligible appear to have varied and included not attending the required employment activ- 
ity, a desire to discontinue involvement with a government program, and a lack of knowledge or 
understanding about eligibility requirements. In addition, despite program services, it appears 
that job loss continued to be an issue, with some ERA participants losing jobs before they be- 
came eligible for the stipend. Notably, among those who did receive a stipend, many continued 
to receive it on an ongoing basis. In Corpus Christi and Fort Worth, almost half of those who 
received a stipend received it 1 1 or more times. 

• Corpus Christi implemented the program most smoothly, and the other 
sites adopted some of its strategies over time. 

Program practices in Corpus Christi included designating case managers to work only 
with individuals who were receiving stipends and developing strong postemployment services, 
including regular site visits to employers, in-house support groups to meet the stipend participa- 
tion requirement, and specific performance measures for staff. These innovative practices reflect 
the very strong management team that the Corpus Christi program had in place during the early 
phases of the project. The Fort Worth program struggled for a good portion of the study period, 
but it made significant improvements when a new manager was hired, including implementing 
more structured job search services and making regular employer site visits. The Houston pro- 
gram moved the most slowly in getting key ERA components — particularly postemployment 
services — off the ground. 

• Compared with Choices, the ERA programs in Corpus Christi and Fort 
Worth increased the proportion of ERA group members who received 
retention and advancement services. 

The ERA retention and advancement services included help finding a better job while 
working, enrolling in life skills classes while working, and career assessment. Despite the in- 
creased participation in these services, the overall level of receipt is somewhat lower than ex- 
pected: Only about 40 percent of survey respondents from the ERA program who were working 
reported that they had received such assistance. This could possibly reflect that — because 
some individuals did not receive the stipend and more intensive retention and advancement ser- 
vices until they had spent more than a year in the program — this activity was not fully captured 
by the ERA 12-Month Survey, which was used to measure service receipt. 

In Corpus Christi and Fort Worth, the ERA program also resulted in significantly more 
contacts with case managers after participants entered the study, but, compared with Choices, 
the differences were not large. In general, participation rates in employment-related services 
were relatively high in all the Texas sites for both the ERA group and the Choices group. 
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Key Findings on Program Impacts 

Table ES.l summarizes the impacts of the Texas ERA program. Administrative records 
are used to examine whether ERA’s work incentive and pre- and postemployment services trans- 
lated into improved employment rates, job retention and advancement, and total income — and 
into reductions in public assistance receipt. Unemployment insurance (UI) wage data and public 
assistance payment records are the primary sources for creating outcomes of employment, earn- 
ings, TANF, and food stamps and for estimating impacts on these outcomes. The main cohort for 
this report includes study participants who were randomly assigned from October 2000 through 
June 2002 (N = 4,288). This represents three-quarters of the eventual sample that will be analyzed 
in Texas. UI records are available for two years after program entry, and welfare and food stamp 
receipt are available for one and a half years. The report’s key impact findings follow. 

• ERA did not produce consistent or large effects on employment or earn- 
ings outcomes. The ERA programs in Corpus Christi and Fort Worth, 
however, led to modest increases above the control group averages on 
some measures of employment and employment retention. The program 
in Houston had no effect on employment and earnings. 

Outcomes for the control group represent what would have happened in the absence of 
the ERA program. In any given quarter, approximately half of control group members were 
employed. On average, they earned only $8,000 over the full two-year follow-up period. (This 
average includes zeroes for those who were not working.) 

ERA did not increase average earnings over the two-year follow-up period. While ERA 
increased the percentage ever employed in Fort Worth, it did not increase measures of employ- 
ment retention in any of the sites. This suggests that the ERA stipend did not increase employ- 
ment and was paid mostly to those who would have worked anyway. 

Yet the two-year impacts in Corpus Christi mask some modest impacts that were evi- 
dent after one year. In Year 1, ERA increased average quarterly employment by over 3 percent- 
age points and increased the proportion of ERA group members who were employed for four 
consecutive quarters — a key measure of employment retention — by over 4 percentage points 
(not shown). During Year 2, however, these effects were no longer statistically significant. Pro- 
gram impacts did not increase over time, as might have been expected. 

The impacts also do not reflect variations in stipend receipt. In Corpus Christi, only 
about 16 percent of the ERA group received a stipend in Year 1 (not shown). In Year 2, ap- 
proximately 24 percent of the ERA group received a stipend, but the impacts on employment 
retention measures were no longer significant. 
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The Employment Retention and Advancement Project 
Table ES.l 

Years 1-2, Impacts on Ul-Covered Employment and Earnings 

Texas 
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In Fort Worth, ERA increased employment by 4 percentage points over the control 
group’s average of 76 percent over the two-year follow-up period. This impact was larger in 
Year 2, which may reflect the strengthening of the ERA program in that site, as noted above. In 
Year 2, ERA group members were 6 percentage points more likely to have been employed than 
control group members. However, there was no effect on measures of job retention. 

For the most part, what impacts there were are not large or consistent. The weak im- 
pacts may be partly attributable to the fact that the ERA program was measured against a group 
of sample members who were engaged in a relatively strong welfare- to-work program. 6 It 
should be noted that, by the end of Year 2, impacts on employment (in Corpus Christi) and on 
earnings (in Fort Worth) emerged (not shown). These impacts had not been evident in the pre- 
ceding quarters, which makes it difficult to know whether they will persist into Year 3. 7 

• Due to the stipend, ERA increased total income in Corpus Christi but 
not in the other Texas sites. For the most part, ERA had no effect on re- 
ceipt of TANF or food stamps. 

Table ES.2 shows that the ERA program increased total income in Corpus Christi by 
$604 over the control group’s average of $1 1,247. Half of this increase is attributable to the sti- 
pend. Thus, it appears that the primary effect of the ERA stipend was to increase income (since 
the program did not encourage employment beyond what would have happened in the absence 
of the program). 

• While the two-year impacts of the Texas ERA program have been weak, 
it is too early to be sure that the program has failed to achieve its goals. 

In particular, because effects seemed to emerge at the end of Year 2 among the full 
samples in Corpus Christi and Fort Worth, and because the Year 3 impacts in Corpus Christi 
among an early cohort look positive, these results should not be considered the final word on 
the Texas ERA program. In addition, most stipend recipients began receiving the stipend in 



6 Since the Texas ERA program began at the preemployment phase but was most different after sample 
members became employed, the impacts may have been slightly smaller than if the evaluation had been con- 
ducted only among sample members who were employed at the time of random assignment. 

7 Analysis from an early cohort indicates that the impacts on employment may have strengthened in Year 
3. Employment and earnings were examined for sample members, randomly assigned from October 2000 
through June 2001, for whom an additional year of follow-up data from administrative records was available. 
In Corpus Christi, the three-year impacts on employment among this early cohort were statistically significant 
throughout the third year of follow-up. By the end of Year 3, impacts for this cohort approached 10 percentage 
points and were statistically significant at the 1 percent level. It is unclear why the impacts were stronger 
among this cohort. It is also unclear why the impacts strengthened later in the follow-up period. It should be 
noted that the sample sizes for this cohort are rather small (N = 668). Among the early cohorts in Fort Worth 
and Houston, there were no statistically significant effects in Year 3. 
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Year 2, and some were still receiving it in Year 3. There is a possibility that the impacts might 
improve later in the follow-up period. MDRC will continue to monitor the impacts of the ERA 
program in Texas. 



Conclusions 

The Texas ERA program is one of 15 being studied as part of the ERA project, and re- 
ports over the next two years will present results for other programs. MDRC will continue to 
track sample members in Texas and will make public longer-term results when they are avail- 
able. As the study continues to generate information, more definitive conclusions will be possi- 
ble. However, some preliminary conclusions can be drawn based on the results in this report. 

• Employment retention and advancement among welfare recipients re- 
mains a goal that is difficult to achieve. 

For the most part, the ERA program in Texas has been unsuccessful thus far in increas- 
ing employment retention and advancement — even in Corpus Christi, the city that most 
strongly implemented the program’s components. Impacts for certain subgroups and cohorts do 
suggest that, under certain circumstances, the program can have a modest effect on these out- 
comes. While it is too early to be sure, the results suggest that, in states with already-strong wel- 
fare-to-work programs, there are likely to be limited gains in “upgrading” to a program that fol- 
lows the Texas ERA model. 

• It is critical to consider the design and marketing of financial incentives. 

Past studies have found that programs providing a financial incentive to encourage work 
among welfare recipients — such as the Minnesota Family Investment Program (MFIP) and the 
Canadian Self-Sufficiency Project (SSP) — have had large effects on increasing employment, 
earnings, and job stability and on reducing poverty. 8 The weaker effects of the Texas ERA pro- 
gram may reflect several factors. First, qualifying for the financial incentive involved a series of 
steps, including working longer than four months, working enough hours, submitting the neces- 
sary documentation, and attending a monthly employment-related activity. Moreover, the more 
intensive postemployment case management (such as employer site visits) did not generally start 
until the stipend phase of the program — after an individual had been working for four months. 
These factors may have limited the number of individuals who became eligible for the stipend and 
may have resulted in the stipend’s going to those who would have begun to work and stayed em- 
ployed without the stipend. While this is not necessarily a negative outcome (the stipend did in- 
crease income for these families), the main goal of the stipend was to increase employment stabil- 



8 Berlin (2000). 
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ity. In contrast, the financial incentive in the MFIP program — which was provided as part of the 
welfare grant, as an earned income disregard — was automatically provided to all those who went 
to work, and it took effect immediately after the recipient became employed. 

Second, while the Texas sites made a good-faith effort to market the ERA stipend, that 
may not have been sufficient. The financial incentive in the Canadian SSP program was also 
voluntary and required individuals to work 30 hours per week, but SSP had nearly universal 
participation among those who became eligible. In contrast to ERA, SSP had a more compre- 
hensive marketing strategy, including a one-on-one orientation session dedicated exclusively to 
a discussion of the financial benefits of the incentive. Although marketing of the ERA stipend in 
Texas was a solid effort that grew stronger over time, information about the stipend was often 
provided along with a range of other information about program requirements and services. 
Marketing the ERA stipend may have been particularly challenging, given that receipt of the 
stipend could seem far in the future to some participants and was contingent on achieving sev- 
eral outcomes in addition to becoming employed. 

Finally, it is important to consider that the magnitude of ERA' s impacts in Texas was 
likely affected by the control group program. Choices was a relatively strong work-focused wel- 
fare-to-work program, which may have produced a more difficult comparison group to “beat” 
than the control groups in MFIP and SSP. 
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Chapter 1 

Introduction 

Overview of the National ERA Project 

For over a decade, policymakers and program operators have struggled to learn what 
kinds of services, supports, and incentives are best able to help low-income working parents 
retain steady employment and move up to better jobs. This issue has assumed greater urgency in 
the wake of the 1990s welfare refonns, which made long-term welfare receipt much less feasi- 
ble for families. Yet, although a great deal is known about alternative approaches to job prepara- 
tion and placement, there is still relatively little hard evidence about effective strategies to pro- 
mote employment retention and advancement. Previous studies on retention and advancement 
efforts — notably, the Post-Employment Services Demonstration (PESD), a four-site project 
that tested programs providing follow-up case management to welfare recipients who found 
jobs — generally failed to improve employment retention. 1 

The Employment Retention and Advancement project was designed to improve on past 
efforts in this area by identifying and testing innovative models designed to promote employment 
stability and wage progression among welfare recipients or other low-income groups. The project 
began in 1998, when the U.S. Department of Health and Human Services (HHS) issued planning 
grants to 13 states to develop new programs. The following year, MDRC was selected by HHS to 
conduct an evaluation of the ERA programs. 2 From 2000 to 2003, MDRC and its subcontractor, 
The Lewin Group, worked closely with the states that had received planning grants — and with 
several other states — to mount tests of ERA programs. MDRC, Lewin, and Cygnet Associates 
also provided extensive technical assistance to some of the states and program operators, since 
most were starting the project from scratch, with no proven models on which to build. 

Ultimately, a total of 15 ERA experiments (also called “tests”) were implemented in 
eight states, including Texas. Almost all the programs target current or former recipients of 
Temporary Assistance for Needy Families (TANF) — the cash welfare program that mainly 
serves single mothers and their children — but the ERA program models are extremely diverse. 
One group of programs targets low-wage workers and focuses strongly on advancement. An- 
other group targets individuals who are considered “hard to employ” and aims primarily to 
place them in stable jobs. Finally, a third group of programs has mixed goals and targets a di- 
verse set of populations, including former TANF recipients, TANF applicants, and low-wage 
workers in particular finns. Some of these programs initiate services before individuals go to 

Rangarajan and Novak (1999). 

2 The U.S. Department of Labor has also provided funding to support the ERA project. 
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work, while others begin services after employment. Appendix Table A.l describes each of the 
ERA programs and identifies its goals and target populations. 

The evaluation design is similar in most of the sites. Individuals who meet ERA eligi- 
bility criteria (which vary from site to site) are assigned, at random, to a program group — also 
called “the ERA group” — or to a control group. Members of the ERA group are recruited for 
the ERA program (and, in some sites, are required to participate in it), whereas members of the 
control group are not eligible for ERA services. The extent and nature of the services and sup- 
ports available to the control group vary from site to site, but it is important to note that, in most 
sites, the ERA program is not being compared with a “no services” control group. The random 
assignment process ensures that any differences in outcomes that emerge between the two re- 
search groups during the follow-up period can be confidently attributed to the ERA program, 
rather than to differences in the characteristics of people in the groups. To track both groups 
over time, MDRC is using surveys and administrative records (data on quarterly earnings in 
jobs covered by unemployment insurance and records of TANF and food stamp payments). 



The Texas ERA Program 

Origins and Goals of the Texas ERA Program 

The Texas ERA program was designed to promote job placement, retention, and career 
advancement as well as to reduce recidivism for TANF applicants and recipients. The program 
provided both pre- and postemployment services and targeted a population applying for or re- 
ceiving cash assistance, most of whom were not working when they entered the program. To 
encourage employment retention and advancement, the Texas ERA program included a 
monthly stipend of $200 for working TANF leavers. The ERA evaluation in Texas was con- 
ducted in three sites: Corpus Christi, Fort Worth, and Houston. 3 

The Texas ERA program was developed by the Texas Department of Human Services 
(DHS), in coordination with the Texas Workforce Commission (TWC). DHS was primarily 
responsible for determining eligibility and overseeing the TANF cash assistance program, while 
TWC managed TANF employment services. TWC oversaw employment services for the ERA 
program as well as for Choices — the standard welfare-to-work program in the state that pro- 
vides employment-related services to TANF recipients. 

The Texas ERA model was explicitly designed to improve on the poor performance of 
past retention and advancement programs, primarily PESD, as well as to enhance the services 

3 The ERA program was also operating in Abilene, but this site was not included in the evaluation because of 
inadequate sample sizes. Although the cities are called “sites” in this report, Texas counts as a single ERA site. 
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provided under Choices. Developed in 1999, the ERA program design grew out of a concern 
with the level of “recycling” in the TANF caseload in Texas. Because of the state’s low grant 
levels, 4 most individuals will leave welfare when they find a job (after an earnings disregard 
period). However, mirroring the experiences of other states and localities, state administrators 
found that many of these individuals worked at low wages with poor benefits and ended up re- 
turning to the rolls. A secondary goal of the Texas ERA program was to increase TANF recipi- 
ents’ participation levels in preemployment services and their overall employment levels. 

The Texas ERA Model 

The Texas ERA program was designed to provide both pre- and postemployment ser- 
vices to TANF applicants and recipients. As designed, the program had three key features: 

• Preemployment job search and team-based case management services. 

For most participants, the ERA program initially provided job search, job 
readiness, and case management services. Because postemployment case 
management services had little effect on employment outcomes in past reten- 
tion and advancement studies, Texas redesigned these services by having 
them begin at the preemployment stage and using a team-based approach. 

The goal was to involve partners from multiple agencies (including DHS, lo- 
cal workforce staff, and organizations working to prevent substance abuse or 
domestic violence) that had expertise in addressing specific employment- 
related barriers. Key case management services included employment as- 
sessment, goal setting and career planning, support services, resolution of 
employment barriers, and job search assistance. 

• Stipend for welfare leavers. The ERA program provided monthly stipends 
of $200 per month to participants who have left TANF, were employed for a 
minimum of 30 hours per week, and participated in a postemployment “ad- 
vancement” activity. The stipend was also available to those who combined 
15 hours a week of employment with an education and training activity that 
lasted 15 hours per week. The stipend was available after a four-month 
earned income disregard, whereby 90 percent of earnings were disregarded 
in calculating the TANF grant. 5 Individuals could receive the earned income 
disregard once in 12-month period (but the four months did not have to be 
used consecutively within this time period). There was a lifetime limit of 12 



4 The maximum grant amount for a single parent who has two children is $203. 

^Depending on participants’ earnings, the amount of the residual TANF grant combined with the earned 
income disregard was roughly equivalent to or somewhat less than the $200 stipend. 
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stipends (stipends also did not have to be used in consecutive months). The 
stipend was included in the ERA model based on other studies that found 
similar earnings supplements to be effective in encouraging job retention and 
increasing earnings (see Box l.l). 6 

• Intensive postemployment services. The ERA program provided a com- 
prehensive set of postemployment services, which could include assistance 
with job-related problems and support services, monitoring job performance 
and issues through regular site visits to employers, rapid reemployment assis- 
tance for participants who lost jobs, and support in meeting the requirements 
of the stipend. ERA postemployment services could continue for as long as 
an individual was eligible for the stipend. 

Immediately following an eligibility or recertification interview for TANF (but, in the 
case of applicants, before they were approved for cash assistance), individuals were randomly 
assigned either to the ERA program or to the Choices program. Those who were assigned to 
ERA were introduced to the program and were then required to attend an orientation to the 
TANF program before being approved for cash assistance. Once these individuals were ap- 
proved for TANF, they were then engaged in ERA program services. Individuals who were not 
approved for TANF were not eligible for the ERA program. 

Participation in the ERA program was mandatory for most individuals, meaning they 
faced a reduction of termination of TANF benefits for noncompliance with program rules. 
Some individuals were not required to participate and did not face sanctions for noncompliance 
— this is known as being “exempt” — if they had a child younger than age 1 , were ill or dis- 
abled, or were caring for a disabled family member. Exempt individuals were eligible for all 
components of the ERA program, and program staff strongly encouraged them to participate. 

Once assigned to the ERA program, individuals completed an assessment; then they gen- 
erally participated in job search and, if they did not find a job, community service (a volunteer 
position in a nonprofit or public organization) and also received case management services. Once 
participants were employed, they received postemployment services, and — after leaving TANF 
and after receiving the earned income disregard — they become eligible for the monthly stipend. 

The Counterfactual: What Is ERA Being Compared With? 

As discussed above, members of the control group were not eligible for ERA services 
and were assigned to participate in Choices, the state’s standard welfare-to-work program. 



^Berlin (2000). 
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Box 1.1 



The Design of Programs Using Financial Incentives 
to Promote Work 

The Texas ERA program was designed in part to build on the results of other programs that 
used financial incentives to encourage work — particularly, the Minnesota Family Investment 
Program and the Canadian Self-Sufficiency Project. As described below, these programs and 
the Texas ERA program had some similarities but also some important differences. 

Minnesota Family Investment Program (MFIP) 

MFIP, which began operating in 1994, allowed cash assistance applicants and recipients to re- 
main eligible for welfare until their income reached 140 percent of the poverty line. Like the 
Texas ERA program, MFIP operated as part of the state’s welfare-to-work system in several 
counties. Under MFIP, the financial incentive was provided as a generous earned income dis- 
regard and was automatically received by all welfare recipients who went to work and re- 
mained on assistance. This resulted in an increase in income of approximately $150 to $250 per 
month, depending on wages and hours worked. Recipients who were not working at least 30 
hours per week were required to participate in job search and other work preparation and train- 
ing programs. Overall, an experimental evaluation of the program showed that MFIP produced 
relatively large increases in employment, earnings, income, and job stability and reductions in 
poverty. (Control group members were not required to participate in employment services and 
faced a sharp reduction in benefits as their earnings increased.) 

Canadian Self-Sufficiency Project (SSP) 

Operating between 1992 and 1999, SSP was a demonstration project designed to test a work- 
based alternative to welfare. It paid a substantial monthly earnings supplement for up to three 
years to single-parent long-term welfare recipients who worked full time (at least 30 hours a 
week). The supplement equaled half the difference between a participant’s earnings and an 
“earnings benchmark” (which equaled $30,000 or $37,000, depending on the site). After taxes, 
SSP made most families better off by $3,000 to $7,000 per year than if they worked full time 
and remained on cash assistance. Sponsored by the Canadian government, SSP was operated 
outside the welfare system by private agencies in parts of two provinces. Participation was vol- 
untary, but recipients could not receive welfare benefits and earnings supplement payments at 
the same time. A person could sign up for the supplement if she found full-time work within a 
year of enrollment. An experimental evaluation of SSP found that it increased full-time em- 
ployment and earnings, reduced poverty, and resulted in more stable employment and wage 
growth over time. (The control group was eligible for cash assistance and received no extra fi- 
nancial incentive to work.) 

Comparison of the Texas ERA Program with MFIP and SSP 

The Texas ERA stipend was similar to the MFIP incentive in that it was provided through the 
cash assistance system and at a similar financial level. However, unlike MFIP’s incentive, the sti- 
pend was not provided automatically to all individuals who became employed and stayed on as- 
sistance. ERA participants had to meet certain requirements in terms of hours worked, complet- 
ing the four-month earnings disregard, and participating in a monthly employment activity. The 
SSP incentive was more generous than the ERA benefit, and the program operated outside the 
welfare system. However, like ERA, SSP required individuals to take the initiative to receive the 
supplement, and they had to work a certain number of hours of work in order to be eligible. 
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Although Choices also provided pre- and postemployment services to TANF recipients, Table 
1 . 1 shows that there were important differences between the two programs, particularly in tenns 
of postemployment services: 

• Preemployment services. These services were similar under both programs. 

As in ERA, individuals who were assigned to Choices completed an assess- 
ment and participated in job search or — if they did not find employment — 
community service. Choices also provided case management services, but 
these generally did not involve multiple agencies and focused less on longer- 
term career planning. Like ERA, Choices services were mandatory for most 
TANF recipients, although the program encouraged those who were exempt 
to participate as well. 

• Postemployment services. Choices participants were not eligible for die 
montiily $200 stipend. Once employed, however, they were eligible for the 
earned income disregard for four months (for the same amount as in the ERA 
program). In Choices, postemployment case management services generally 
lasted only for die duration of the earnings disregard, whereas services continued 
for up to an additional 12 months in ERA, for those who were receiving the sti- 
pend. Postemployment services in Choices were also less intensive — consisting 
primarily of montiily contacts with participants to verily employment status. 
Members of both groups were eligible for transitional child care subsidies once 
diey left TANF after receiving the earned income disregard. (While receiving the 
disregard, individuals continued to receive child care through TANF.) 

Because of the job search and case management services, participation requirement, and 
support services, the control group in the ERA evaluation received a relatively strong set of ser- 
vices, compared with other experimental studies of programs involving financial incentives. 
Box 1.1 describes two such programs: the Minnesota Family Investment Program (MFIP) and 
the Canadian Self-Sufficiency Project (SSP). 

Characteristics of the Texas ERA Sites 

The three Texas sites in the ERA evaluation differ significantly in terms of their local 
economy and the demographic characteristics of the ERA target population. Corpus Christi, a 
midsize city located on the Gulf Coast, has an economy based on petroleum, shipping, and tour- 
ism. Economic conditions have been relatively poor: In 2003, the unemployment rate was 6.7 
percent — having increased from 6.3 percent in 2000, when the study began. 7 



7 U.S. Department of Labor, Bureau of Labor Statistics (2004). 
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The Employment Retention and Advancement Project 
Table 1.1 

Comparison of Services Provided in the Texas ERA Program 
and in the Control Group Program (Choices) 



Services 


ERA Program 


Choices 


Preemnlovment job search services 

Strong focus on immediate 
employment 


Yes 


Yes 


4 weeks of job search followed by 
community service for those who did 
not find jobs 


Yes 


Yes 


Preemnlovment case management 

Team based 


Yes 


No 


Assessment and monitoring 
participation 


Yes 


Yes 


Barrier identification and career 
planning 


Yes 


Limited 


Support services including child care 
and transportation 


Yes 


Yes 


Financial incentives 

For those who find jobs, 4-month 
earned income disregard period 
where 90 percent of earnings are 
disregarded in calculating the welfare 
grant 


Yes 


Yes 


$200 per month stipend for TANF 
leavers who are working full time or 
working part time combined with 
school 


Yes 


No 


Postemplovment services 

Case management during earned 
income disregard period 


Yes 


Limited 


Intensive case management during 
stipend period including employer 
site visits and rapid re-employment 
services 


Yes 


No 


Transitional child care for TANF 
leavers 


Yes 


Yes 
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In contrast, Fort Worth is the sixth-largest city in Texas. 8 Located in Tarrant County, it 
is a major mercantile, commercial, and financial center providing agribusiness and wholesale 
services to much of west Texas. The city also has diversified manufacturing and significant 
tourism industries. In December 1999, the unemployment rate was 2.8 percent, and it increased 
to 6.4 percent by 2003. 9 The child poverty rate in Forth Worth was very low compared with the 
rest of the state, while median household income was comparatively high. 

Houston is not only the largest city in Texas but also the largest city in the southern 
United States and the fourth-most-populous city in the nation. Houston is regarded as a major 
port and corporate management center, and the city is home to many businesses. Houston’s 
highly industrialized economy is based on petrochemicals, medical research and health care 
delivery, high technology (including computers and aerospace), manufacturing and distribution, 
and related service industries, among others. 10 As of December 1999, the unemployment rate 
was 4.2 percent, but it increased to 6.9 percent in 2003. The child poverty rate was similar to 
rates in the rest of the state, while median household income was relatively high. 

The ERA Target Population 

Table 1.2 shows selected characteristics of ERA group members at the point they en- 
tered the study in each of the three Texas sites. As expected, given that the program targets 
TANF applicants and recipients, very few ERA sample members — ranging from 4 percent to 
9 percent across the sites — were employed at the time of random assigmnent. The low levels 
of employment at this point were also due to the relatively low grant levels in Texas (see be- 
low), which meant that few sample members combined work and welfare. In addition, nearly 1 5 
percent had not worked in the past three years, and roughly 40 percent had worked a year or less 
in this same time period. The majority of those who worked did so “mostly full time.” 

The sample members across the sites varied the most in terms of race/ethnicity. In Cor- 
pus Christi, the sample is primarily Hispanic; in Fort Worth and Houston, the majority are Afri- 
can-American, though Houston also has a sizable Hispanic population. Over half the sample 
members in Houston and Corpus Christi did not have a high school diploma or General Educa- 
tional Development (GED) certificate when they entered the study, while slightly less than half 
the sample members in Fort Worth were in this group. 

Across all the Texas sites, sample members were generally not long-term TANF recipi- 
ents: About one-third had no history with Aid to Families with Dependent Children (AFDC) or 
TANF, and over 40 percent had received assistance for fewer than two years. Over 80 percent 

8 Texas State Library and Archives Commission (2000). 

9 U.S. Department of Labor, Bureau of Labor Statistics (2004). 

10 City of Houston (2005). 




The Employment Retention and Advancement Project 
Table 1.2 



Selected Characteristics of Texas ERA Group Members 3 



Characteristic 


Corpus 

Christi 


Fort 

Worth 


Houston 


Gender (%) 


Male 


5.7 


3.8 


2.3 


Female 


94.3 


96.2 


97.7 


Average age (years) 


28.5 


28.6 


28.2 


U.S. citizen (%) 


Yes 


98.7 


99.2 


96.2 


No 


1.3 


0.8 


3.8 


Limited English (%) 


Yes 


3.3 


0.6 


2.8 


No 


96.7 


99.4 


97.2 


Race/ethnicity (%) 


White, non-Hispanic 


15.9 


21.5 


9.3 


Black, non-FIispanic 


8.3 


66.9 


61.9 


American Indian/ Alaska native 


0.1 


0.1 


0.1 


Hispanic 


73.6 


10.8 


27.7 


Other 


2.1 


0.6 


1.0 


Education (%) 


GED 


16.4 


13.6 


15.7 


High school diploma 


25.6 


39.1 


25.7 


Technical/Associate's degree/2-year college 


5.4 


1.7 


2.7 


4-year (or more) college 


0.9 


0.8 


0.7 


None of the above 


51.7 


44.9 


55.2 


Current cash assistance status (%) 


Applicant 


88.3 


81.7 


84.4 


Recipient 


11.7 


18.3 


15.7 


Registration status (%) 


Mandatory 


75.1 


77.8 


77.5 


Exempt 


24.9 


22.2 


22.5 


Total prior AFDC/TANF receipt (%) 


None 


38.8 


35.5 


38.5 


Less than 3 months 


5.7 


6.0 


9.4 


3 months or more and less than 2 years 


34.9 


43.9 


34.9 


2 years or more and less than 5 years 


12.7 


8.1 


9.9 


5 years or more and less than 10 years 


5.7 


5.0 


5.4 


10 years or more 


2.3 


1.6 


2.0 



(continued) 
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Table 1.2 (continued) 



Characteristic 


Corpus 

Christi 


Fort 

Worth 


Houston 


Months employed in last 3 years 


Did not work 


14.6 


13.1 


14.7 


6 or less 


18.3 


22.2 


21.0 


7 to 12 


18.1 


17.4 


22.1 


13 to 24 


21.2 


21.2 


19.7 


More than 24 


27.9 


26.2 


22.5 


Type of employment in last 3 years (among those who worked) (%) 


Mostly part time 


28.4 


18.3 


21.9 


Mostly full time 


56.4 


62.8 


73.3 


Equal amounts part time and full time 


15.2 


19.0 


4.8 


Currently employed (%) 


Yes 


8.8 


4.3 


6.8 


No 


91.3 


95.7 


93.2 


Hours worked per week (among those currently employed) (%) 


10 or less 


7.3 


12.9 


8.9 


1 1 to 20 


36.5 


21.0 


23.2 


21 to 40 


37.2 


40.3 


40.2 


More than 40 


19.0 


25.8 


27.7 


Average hourly wage (among those currently employed) ($) 


6.15 


7.91 


6.83 


Number of children (%) 


0 


1.5 


0.3 


1.2 


1 


44.1 


41.9 


37.1 


2 


28.9 


30.7 


29.4 


3 or more 


25.6 


27.1 


32.3 



Sample size (total = 5,237) 1,723 1,564 1,950 



SOURCE: Texas baseline information sheets. 



NOTES: See Appendix B. 

a This table includes single-parent ERA sample members only. Two-parent families are not included. 



of the sample members were TANF applicants, and about one-quarter were exempt from the 
mandate to participate in program services, usually because they had a child younger than age 1 . 

The External Environment 

The Texas ERA program was developed and implemented during a period when sev- 
eral policy changes were occurring in the state. In 1995, prior to the implementation of ERA, 
responsibility for TANF employment services was shifted by the state legislature from DHS to 
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TWC. In addition, during the early phases of the program, Texas was operating under a waiver 
from the federal welfare reform law that allowed individuals to participate in a wide range of 
employment and training activities in order to meet the federal work participation requirements. 
When the waiver ended in July 2002, Texas was required to follow the federal rules — which 
placed more limitations on the types of activities that counted toward meeting federal participa- 
tion rates — and more strongly emphasized work-focused activities. Across all the Texas sites, 
this resulted in a stronger focus on immediate employment both in ERA and in Choices. 

During the period of the ERA evaluation, TANF caseloads were increasing slightly in 
Texas. From 2000 to 2003, caseloads increased from 342,000 to 370,000 — an increase of 8 
percent. 11 TANF grant levels also increased slightly; the average basic monthly grant for a fam- 
ily of three in Texas went up from $188 in 2000 to $203 in 2003. 12 The state’s TANF grant lev- 
els were among the lowest in the nation. 13 

More recently, there has been a reorganization of agencies involved in the program at 
the state level. In 2004, DHS was merged with a larger umbrella agency that is responsible for a 
range of social services, which changed the state-level management of ERA. In addition, Texas, 
like many other states, recently experienced tighter state budgets. Because of these factors, the 
ERA program ended in August 2004. 



About the Evaluation 

Research Questions 

The ERA evaluation includes two major components: (1) an implementation analysis, 
which studies the way the program operates, and (2) an impact analysis, which assesses what 
difference the program makes relative to the current environment. Contingent on future evalua- 
tion design decisions, a benefit-cost analysis — which will compare the financial benefits and 
costs of the ERA program for participants and for the government budget — may also be con- 
ducted. 



This report focuses on program implementation and early impacts and addresses the 
following questions. 

• Implementation. How did each of the Texas sites execute its ERA program? 

What services and messages did ERA clients in Texas receive? How did case 



"Texas Department of Human Services (2003). 

12 Texas State Services, Legislative Budget Board (2003). 

13 U.S. House of Representatives, Committee on Ways and Means (2005). 
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managers spend their time? How did implementation of the ERA program 
compare with implementation of the Choices program? 

• Participation. Did the ERA program succeed in engaging a substantial pro- 
portion of individuals in its services? What types of services did people re- 
ceive? What proportion received a stipend? How did participation levels in 
ERA compare with levels of participation in Choices? 

• Impacts. Within the follow-up period, did each of the Texas ERA programs 
— relative to the experiences of the control group — increase employment 
and earnings and reduce public assistance receipt? Did individuals’ measured 
income increase as a result of the program? 

The final report on the evaluation of the ERA project will provide program impacts 
covering a longer-follow-up period and may include a benefit-cost analysis. 

The ERA Research Design 

As discussed above, to produce reliable estimates of the effects of the Texas ERA pro- 
gram, the evaluation uses a random assignment research design. The random assignment proc- 
ess ensures that any differences in various outcomes that emerge between the two research 
groups during the follow-up period can be confidently attributed to the ERA program. These 
differences in outcomes are known as impacts. 

The random assignment process began in November 2000 in Corpus Christi and Fort 
Worth and in March 2001 in Houston. Random assignment ended in all the Texas sites in De- 
cember 2002, and the ERA program ended in August 2004. Thus, the amount of time that sample 
members could potentially be exposed to the ERA program ranged from 20 months to 46 months, 
depending on when they were randomly assigned. However, it is unlikely that participants would 
be engaged in ERA for as long as 46 months. Because of differences in program implementation 
across the sites, impacts are presented for each site and are generally not pooled. 

Most sample members were single parents when they entered the study, and this group 
is the focus of the evaluation. Appendix Table E.12 compares single parents who were assigned 
to the ERA group with single parents in the control group, showing that the groups are similar 
along most dimensions. Because program administrators in Texas had an interest in understand- 
ing the effects of the program for two-parent families, this group was included in the random 
assignment process in Corpus Christi. However, as discussed below, sample sizes were too 
small to conduct most of the analyses. 

When a TANF applicant or recipient met with a DHS staff person to complete an eligi- 
bility or recertification interview, staff referred those who they believed would be certified (or 
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recertified) for TANF for random assignment in the ERA evaluation. After staff completed 
baseline paperwork for each individual — recording such standard client characteristics as edu- 
cational background and welfare history — individuals were randomly assigned to either the 
ERA or the Choices group. 14 

Both ERA and control group members were assigned to attend a workforce orientation, 
which was required in order to be certified for TANF. (Individuals who did not attend the orien- 
tation could not receive TANF or participate in either ERA or Choices.) The sites operated 
separate orientations for ERA and Choices, and the content reflected the differences in the pro- 
grams. As in all welfare-to-work programs, many individuals who were referred to ERA or to 
Choices did not attend a workforce orientation or a program activity — some found employ- 
ment on their own; some were not approved for receiving TANF; and some chose not to par- 
ticipate in program services (thereby facing possible sanctions). The attrition between TANF 
application and certification for both groups is examined in this report. 

Data Sources 

Most of the report’s findings cover a two-year follow-up period. The data sources and 
the sizes of the samples for each type of analysis are described below and in Table 1 .3. 

Unemployment Insurance, TANF, and Food Stamp Records Data 

Employment, earnings, and public assistance impacts were computed using automated 
records data from the Texas unemployment insurance (UI) system and administrative records 
from the TANF and Food Stamp Programs. One year’s records of UI, TANF, and food stamp 
receipt are available for all the sample members in all the Texas sites. The sample sizes reported 
in Table 1.3 are for single parents, who are the primary focus of the analysis. As discussed 
above, 178 two-parent families were also randomly assigned, primarily in Corpus Christi. 

The primary sample used in the impact analysis — known as the “report sample” — in- 
cludes single parents for whom two years of follow-up data are available. This includes those 
who were randomly assigned between October 2000 and June 2002 in Corpus Christi and Fort 
Worth and between March 2001 and June 2002 in Houston. The report sample represents 76 
percent of the full sample in Corpus Christi, 74 percent in Fort Worth, and 93 percent in Hous- 
ton. Three years of data are available for an early cohort assigned through June 200 1 in all the 
sites. This cohort represents 39 percent of the full sample in Corpus Christi, 45 percent in Fort 
Worth, and 35 percent in Houston. 



14 Across all the sites, 50 percent were assigned to the ERA group, and 50 percent were assigned to the 
control group. 
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The Employment Retention and Advancement Project 



Table 1.3 

Overview of Report Sample Sizes, by Site and Research Group 



Site 


Report Sample 


Early Cohort 


12-Month Survey 


Corpus Christi 

Random assignment period 
ERA group 
Control group 
Total sample size 


Oct. 2000 - June 2002 
656 
653 
1,309 


Oct. 2000 - June 2001 
337 
331 
668 


Jan. 2002 - June 2002 
141 
149 
290 


Fort Worth 

Random assignment period 
ERA group 
Control group 
Total sample size 


Oct. 2000 - June 2002 
577 
586 
1,163 


Oct. 2000 - June 2001 
347 
363 
710 


Sept. 2002 - Dec. 2002 
92 
96 
188 


Houston 

Random assignment period 
ERA group 
Control group 
Total sample size 


Mar. 2001 - June 2002 
904 
912 
1,816 


Mar. 2001 - June 2001 
340 
333 
673 


Jan. 2002 - June 2002 
150 
147 
297 



The ERA 12-Month Survey 

Clients’ experiences in the program and their service receipt — as well as some meas- 
ures of their income and job characteristics — are based on results compiled from a survey ad- 
ministered approximately one-year after random assigmnent to a sample that was evenly split 
between ERA and control group members. The survey sample was selected from adults in sin- 
gle-parent families who spoke either English or Spanish and who were randomly assigned from 
January through June 2002 in Corpus Christi and Houston and from September through De- 
cember 2002 in Fort Worth. 15 This report includes the responses of 775 individuals (290 in Cor- 
pus Christi, 188 in Fort Worth, and 297 in Houston). The response rates for the survey are 82 
percent in Corpus Christi, 75 percent in Fort Worth, and 80 percent in Houston. 



15 In Fort Worth, the survey sample is drawn from a different (and later) random assignment period than 
the report sample. 
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Field Research 



Periodically between 2000 and 2004, MDRC staff interviewed case managers, service 
providers, and program administrators from both ERA and Choices. Information was collected 
about a range of issues, including program goals; the nature of job search, education and train- 
ing, and support services; marketing and administration of the stipend; retention and advance- 
ment services; management philosophies and structure; relationships between organizations 
involved in the program; and enforcement of the participation mandate. 

Data on Stipend Receipt 

MDRC received program tracking data from each of the sites on stipend receipt for the 
full sample of individuals in the ERA group. Stipend data are available for the entire period that 
the Texas ERA program operated, from October 2000 to August 2004. 

Time Study of Program Staff 

MDRC drew on data collected from a two-week time study that was administered to 
ERA staff. The time study collected detailed information on the nature of staff-client interactions 
and on the topics covered in these interactions. In addition, the study collected information on how 
case managers typically spent their time each day. The time study was administered confiden- 
tially, using MDRC-assigned identification numbers to protect the identity of case managers. The 
time study was administered in 2003, from July 14 to 27 in Corpus Christi and Houston and from 
January 29 to February 1 1 in Fort Worth. In Corpus Christi, 73 percent of staff completed the time 
study, while 94 percent of staff in Fort Worth and Houston did so. A total of 22 staff completed 
the time study in Texas: 9 in Corpus Christi, 6 in Fort Worth, and 7 in Houston. 

Telephone Discussions 

Between October 2003 and March 2004, MDRC conducted a special study in which a 
small number of sample members were contacted by telephone to discuss their experiences with 
the ERA postemployment stipend. Drawing from the sample of individuals who completed the 
12-month survey in Corpus Christi and Houston, this sample includes 20 individuals who, as of 
May 2003, (1) had received more than two monthly stipends or (2) had not received a stipend 
but were employed for four or more consecutive months, according to ERA program records. 

Baseline Demographic Data 

Data on clients’ characteristics, such as educational background and welfare history, 
were collected by welfare staff after eligibility interviews and are available for all individuals in 
the research sample. 
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Roadmap of the Report 

As previously mentioned, this report focuses on the Texas ERA program’s implementa- 
tion and impact findings. Chapter 2 further describes the program and its implementation. Chap- 
ter 3 provides infonnation regarding impacts on service receipt. Chapter 4 covers impacts on 
employment, earnings, job characteristics, and other outcomes. 
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Chapter 2 

Implementation of the Texas ERA Program 



In order to interpret the impacts that the Texas Employment Retention and Advance- 
ment (ERA) program had on employment and other outcomes, it is important to understand 
how the program operated and how it was different from what members of the control group 
experienced in Choices, the state’s standard program for recipients of Temporary Assistance for 
Needy Families (TANF). Drawing from field research, program data, and a time study of pro- 
gram staff, this chapter focuses on how the ERA program was implemented in the three Texas 
ERA sites: Corpus Christi, Fort Worth, and Houston. 1 

After a brief summary, this chapter describes how the Texas ERA program was put into 
place and its structure, staffing, and management. It then discusses the program’s services, how 
program staff spent their time, and differences in implementation across the three sites. 



Key Findings 

While the Texas ERA program was intended to be distinct from Choices in terms of 
both pre- and postemployment services, most of the key differences were in the nature of the 
postemployment services, particularly the monthly stipend of $200 for working TANF leavers. 
In all three sites, the Choices program was relatively strong in terms of preemployment services 
— and experienced some improvements over the course of the study period — and the services 
were similar to those provided by ERA. Although team-based case management (involving 
partners from multiple agencies) was an important concept in the development of the ERA pro- 
gram and one that was intended to distinguish it from Choices, in the end the preemployment 
case management services of the two programs were similar and did not generally involve a 
range of organizations. 

For TANF leavers who found jobs, ERA’s retention and advancement services were 
strongest and most distinct from Choices after the four-month earnings disregard period — once 
individuals were receiving a stipend. At this point, participants began working most closely 
with the postemployment staff, typically meeting with them at least once a month. In Corpus 
Christi and Fort Worth, program staff often conducted this monthly meeting at the worksite of 
those who were receiving a stipend — although this did not begin in Fort Worth until later in 
the study period. 



'Although the Texas program operated in various cities that are called “sites” in this report, Texas counts 
as a single ERA site. 



17 




Overall, Corpus Christi achieved the highest stipend receipt rate: Approximately 30 
percent of the ERA group received it. (The rates in the other two sites were lower.) Individuals 
who found jobs and received the entire four-month earned income disregard generally became 
eligible for a stipend — although they may have not received it if they did not work enough 
hours, submit the required documentation, or attend the required activity. Among those who did 
become eligible for the stipend because they worked longer than the disregard period, about half 
received at least one stipend. 

All three Texas sites found that significant effort had to be put into marketing the sti- 
pend, and they made a good-faith effort to do so, particularly over time. Yet there was clearly 
room for improvement. Among the participants who were eligible for the stipend, the reasons 
for not getting it appear to have varied and included not attending the required employment ac- 
tivity, having a desire to discontinue involvement with a government program, and lacking 
knowledge or understanding of the stipend requirements. In addition, despite the program’s ser- 
vices, it appears that job loss continued to be an issue in the ERA program, with some partici- 
pants losing jobs before they became eligible for the stipend. Notably, among those who did 
receive a stipend, many received it on an ongoing basis. 

Corpus Christi implemented the ERA program most smoothly and developed several 
strategies that were adopted by the other two sites over time. These included marketing strate- 
gies for the stipend, designating case managers to work only with individuals who were receiv- 
ing stipends, and developing strong postemployment services (including regular site visits to 
employers, in-house support groups to meet the stipend participation requirement, and specific 
perfomiance measures for staff). These innovative practices reflect the work of the strong man- 
agement team that the program had in place during the early phases of the project. The Fort 
Worth program struggled for a good portion of the study period but made significant improve- 
ments when a new manager was hired and when more structured job search services and regular 
employer site visits were put into place. Houston moved most slowly to launch key components 
of the program — particularly postemployment services. 



The Framework: Structure, Staffing, and Management 

Organizational Structure and Program Funding 

As discussed in Chapter 1, the Texas ERA program was designed by the state’s De- 
partment of Human Services (DHS), in coordination with the Texas Workforce Commission 
(TWC). At the local level, all employment services for TANF recipients are coordinated by lo- 
cal workforce development boards, under the guidance of TWC. Both the ERA program and 
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Choices were funded by TANF, 2 although, for ERA, the workforce boards contracted with DHS 
for funds, whereas resources for Choices are administered directly by TWC. 3 In each of the 
sites, the local workforce development boards contracted with nonprofit organizations to oper- 
ate the ERA and Choices employment programs. 

Each site had some discretion in how it structured the ERA program, although each fol- 
lowed the same general model. Table 2.1 presents the key partners, roles, and staffing arrange- 
ments in the Texas ERA Program. Except in Fort Worth, the nonprofit agency that was under 
contract with the local board to operate the ERA program was also the contractor for Choices; 
for the most part, however, each program had separate staff. In Fort Worth, the workforce board 
contracted with a new organization — the Women’s Center — for some ERA services (primar- 
ily case management) while also using the Choices contractor and staff for other ERA services 
(primarily job search). In Houston and Fort Worth, the ERA and Choices programs operated 
from the same locations, in One-Stop Centers. 

As part of developing a team-based case management approach, each site established 
partnerships with other organizations in operating its ERA program. The partnership in all the 
sites included the local DHS office and an organization working to prevent substance abuse. 
Houston also included a partnership with a domestic violence organization and a nonprofit or- 
ganization that connected participants to training options. In contrast, the Choices program did 
not explicitly develop any partnerships with other organizations. Choices did work with DHS 
staff on eligibility issues, but the ERA programs generally forged stronger linkages with this 
agency. In Corpus Christi, for example, the ERA staff worked with only two DHS eligibility 
staff, whereas the Choices staff had to coordinate with a much larger number of DHS staff. 

Staffing and Training 

ERA services in Texas were provided primarily by case managers who were employed 
by the nonprofit agency operating the ERA program in each site. As shown in Table 2.1, while 
staff responsibilities in the ERA program evolved over time in all the sites, all ended up using a 
specialized case management approach — with one set of staff responsible for preemployment 
case management activities (primarily assessment and monitoring participation); another respon- 
sible for job search, job development, and reemployment assistance (in Corpus Christi and Fort 



2 The exception is the ERA stipend, which was funded by another source — Aid to Families with Depend- 
ent Children (AFDC) sanction resettlement hinds from the U.S. Department of Flealth and Fluman Services 
(HHS). As a result, TANF regulations on the definition of “assistance” do not apply to the stipend. Payments 
that are categorized as assistance are subject to certain rules, primarily the federal five-year time limit. 

3 In 1998, when FIFIS’s Administration for Children and Families (ACF) started the planning phase of the 
ERA project, it issued planning grants to the designated TANF agency in each participating state. At the time, 
DFIS was the designated TANF agency in Texas, although that responsibility now falls to TWC. 
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The Employment Retention and Advancement Project 



Table 2.1 

Organizational Roles, Key Partners, and Staffing Arrangements 
in the Texas ERA Program 





Corpus Christi 


Fort Worth 


Houston 


Lead agency 


Coastal Bend Workforce 
Development Board 


TWC/Workforce 
Development Board 


Houston-Galveston Area 
Council 


Provides TANF 
eligibility services 


DHS 


DHS 


DHS 


Operates ERA 
program 


Workforce- 1 


Women’s Center, CERCO a 


HoustonWorks 


Operates Choices 
program 


Workforce- 1 


CERCO a 


HoustonWorks 


Other ERA 
program partners 


Council on Drug and 
Alcohol Abuse 


Council on Drug and 
Alcohol Abuse 


Employment and Training 
Centers, Inc.; Council on 
Alcohol and Drugs; 
Houston Area Women’s 
Center 


Programs/services 
colocated at ERA 
program 


DHS 


DHS; One-Stop Center 


One-Stop Center; Houston 
Area Women’s Center; 
Employment and Training 
Centers, Inc. 


ERA case 

management 

approach 


Specialized: pre- 
employment case 
management staff; job 
search/job development 
staff; postemployment staff 


Specialized: pre- 
employment staff; job 
search/job development 
staff; postemployment 
staff. Women’s Center 
provides pre- and post- 
employment case 
management; CERCO 
provides job search 
services. 


Started as generalist 
approach but moved to 
specialized: pre- and 
postemployment staff. 



NOTE: 11 In September 2003, CERCO replaced Work Advantage as the contractor for ERA and Choices. 



Worth only); and another responsible for postemployment case management activities (monitor- 
ing individuals’ employment status, issuing stipends, assisting with job-related issues). In all three 
sites, DHS staff — not ERA program staff — were responsible for TANF eligibility issues. 

Table 2.1 shows, however, that there were important distinctions across the sites. Cor- 
pus Christi moved most quickly to establish its staffing structure. Fort Worth used staff from 
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two organizations to provide different services to ERA participants for much of the study period 
— with one organization being focused on case management and the other providing job search 
activities. Houston used a generalized case management approach for much of the study period 
but moved to a specialized approach over time. The size of the programs in temis of staff var- 
ied. Corpus Christi generally had the largest staff, at one point including a dozen case managers 
as well as other administrative staff and a workshop facilitator. The Fort Worth and Houston 
programs were generally smaller, with six to eight case managers. Caseload sizes varied across 
the sites, across type of staff, and over time. All the sites went through some period over the 
course of the study when they were short-staffed and caseloads were higher than intended. 

Although team-based case management was an important concept in the development 
of the ERA program, in the end, ERA case management services did not involve staff from a 
range of organizations. In general, the primary staff providing services were ERA staff from the 
agency that contracted to operate the program, rather than from other organizations. As dis- 
cussed above, several of the programs developed linkages with other organizations to provide 
specialized services, but these were generally used on an as-needed rather than routine basis, 
and few participants ended up accessing them. For example, in Houston, a representative from a 
domestic violence organization was colocated at the ERA program but provided assistance only 
to those who requested it — and few did. Across the sites, staff from organizations to prevent 
substance abuse were initially involved in assessing participants’ needs for these services, but 
they generally did not work with participants on an ongoing basis. By the end of the study pe- 
riod, these organizations had very limited involvement with the ERA program, in large part due 
to perceived limited interest in and use of their services by program participants. 

Staff varied in their previous employment experiences. Some — particularly in the 
Choices programs — had a history of working with welfare-to-work programs. Others did not 
have experience in social services but brought experience from other related fields, such as em- 
ployment placement or temp agencies. DHS held annual training sessions for ERA staff in all 
three sites, covering a range of issues related to the program, including the marketing of finan- 
cial incentives, developing career ladders, and identifying strategies to prevent job loss and 
promote advancement. 

The Choices program used a more traditional case management approach: The same 
case manager worked with a client throughout the client’s time in the program. There were no 
staff who specialized in job search or job development activities or in working with clients after 
they found jobs. 
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Management 



At the local level, the ERA program was generally managed by a program director at 
the local workforce development board, and a program manager was responsible for day-to-day 
operations at the contracted agency. Over the course of the study, there were changes in the 
management of the program in two of the sites. In Fort Worth, a new program manager started 
in fall 2002 and was the impetus for many subsequent program improvements. In Corpus 
Christi, the strong management team that was in place during the early phases of the project and 
was responsible for developing the most innovative initial ERA program moved on to other po- 
sitions and was replaced in summer 2002. 

In addition to overseeing its contracts with the local workforce board in each of the 
sites, DHS also played a management role in the ERA program. DHS set perfomiance targets 
for the individual ERA sites in several areas, including participation, placement rates, retention 
rates, and wage increases. State staff held monthly conference calls with the sites to review per- 
formance on these measures and to provide opportunities for sharing best practices. DHS staff 
also conducted regular site visits to assess operations and provide technical assistance. 

Corpus Christi was the only site to develop a comprehensive performance management 
system for staff, based on measures similar to those developed by DHS (see Box 2.1). Although 
the staff performance measures in Corpus Christi evolved somewhat over time, both manage- 
ment and line staff took these performance measures seriously. The other sites generally did not 
have specific performance measures for staff — particularly measures related to job retention 
and advancement issues. For example, Houston did have some job placement measures for staff 
but had none related to postemployment services or stipend receipt. The Choices programs also 
did not include any comprehensive performance systems for staff, although some did require 
staff to meet a certain number of job placements. 



The Texas ERA Program’s Services and Messages 

While there were some variations across the three sites, Figure 2.1 illustrates the typical 
path of individuals through the Texas ERA program. The rest of the section discusses the ser- 
vices that were provided, with attention given to the differences across sites. (Chapter 3 presents 
quantitative infomiation on the ERA and control groups’ use of employment-related services.) 

Preemployment Services: Job Preparation Services 

As noted in Chapter 1 , individuals were randomly assigned to the ERA or the Choices 
group at their TANF application or recertification interview at the local DHS office. Those who 
were assigned to ERA attended a group “workforce orientation” that was required in order to 
become certified for TANF. Once certified, individuals were referred to the ERA program. If 
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Box 2.1 



Corpus Christi’s Performance Management System 



Throughout the course of the study, the Corpus Christi ERA program used specific perform- 
ance measures to assess staff performance. Although these changed over the course of the 
study, they were focused on key tasks, including participation levels, job placements, retention 
rates, wage increases, employer site visits, and payment of stipends. Because staff were special- 
ized in terms of their job responsibilities, performance measures varied for different types of 
staff. For most of the study period, staff who were responsible for managing preemployment 
services for a case were expected to have at least two-thirds of their caseload actively partici- 
pating in program activities at any time. Staff who were responsible for helping participants 
find jobs were evaluated primarily on the basis of job placement rates and average wage 
placement. Postemployment services staff, who worked with participants once they were re- 
ceiving a stipend, were evaluated based on wage rates and participants’ retention after 1, 6, 12, 
and 1 8 months of employment. They were also required to contact each participant monthly, 
meet with each participant quarterly, and conduct at least 25 employer site visits per month. 
Later in the study period, performance measures for postemployment staff changed somewhat; 
each month, staff were expected to issue 50 stipends, make 38 employer site visits, and have 
five participants with a wage increase. 

Each staff person in Corpus Christi met individually with his or her supervisor for a monthly 
review, which was designed to discuss progress made toward meeting performance bench- 
marks. If a staff member was not meeting benchmarks, the supervisor reviewed cases with him 
or her to find ways to make improvements. 



individuals did not attend the workforce orientation or meet other eligibility requirements, they 
could not receive TANF or ERA services (including the stipend). As a result, and as discussed 
further below, some individuals in the sample were not certified for TANF and could not par- 
ticipate in the ERA program. 

Separate workforce orientations were held for ERA and Choices; the orientations were 
usually held on a different day than the eligibility or recertification interview at the DHS or the 
ERA/Choices office. In the orientations, individuals heard about the purpose of the program and 
its services and requirements, and they were scheduled to meet with an ERA case manager. 
While both orientations emphasized the importance of work and the program requirements, the 
ERA orientation included more information on the stipend and other services unique to ERA. 
At the workforce orientation, clients were referred for services at the ERA program. 

Across the three sites, job placement was the primary goal for program participants, and 
the preemployment services were focused on this objective. As the first step in the program, 
participants generally attended a job search workshop of several days (the duration of the work- 
shop varied by site and over time). The workshop covered a range of job search activities, 
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The Employment Retention and Advancement Project 

Figure 2.1 

Typical Paths of Individuals Through the Texas ERA Program 




NOTE: “These individuals remain in the study sample. 
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including resources, resume development, and interviewing techniques, and it included a dis- 
cussion and marketing of the postemployment stipend. As part of the job search workshop, par- 
ticipants met individually with their case manager, an assessment was conducted, and support 
services were arranged. (Those needing child care were referred to a contractor who handled 
these arrangements.) 

The assessments generally focused on reviewing education and work history and ad- 
dressing needs for support services. There was some focus on identifying long-term career goals 
and pathways as part of the assessment — particularly in Corpus Christi. The assessment also 
included a test of basic reading and math skills and screenings for learning disabilities and for 
mental health and substance abuse issues. During the early stages of the program, Fort Worth 
had a very strong focus on the assessment component. This was often a multistep process in- 
volving various staff, and the process had the effect of limiting the number of individuals who 
received other program services. Throughout the study period, Houston also included an addi- 
tional in-depth assessment for many participants (which involved an extensive interview with a 
staff person who was a licensed psychologist). 

The Texas ERA program in all the sites had a strong focus on immediate employment, 
particularly when the federal waiver from participation requirements ended; it had allowed indi- 
viduals to engage in a wide range of employment and training activities in order to meet the 
federal work participation requirements. After the waiver, an individual’s initial activity was 
generally a four-to-six-week search for employment. Before the waiver expired, however, al- 
though job search was still the primary initial activity, the sites allowed more flexibility and in- 
cluded such activities as obtaining a General Educational Development (GED) certificate. Dur- 
ing the job search phase, participants were typically expected to make a certain number of em- 
ployer contacts each month and to bring a completed job log to their weekly meeting with the 
case manager. Staff did explore clients’ interests and, if possible, helped them find jobs that fit 
those interests. But — largely because of the emphasis on achieving the federal participation 
rates — staff usually encouraged participants to take a job quickly. 

The sites varied in how they handled job development — or locating potential job open- 
ings for ERA participants. The Corpus Christi and Fort Worth programs designated specific 
staff to focus on job development and placement, whereas the Houston program gave this re- 
sponsibility to individual case managers. During the early phases of the program in Fort Worth, 
difficulties arose in coordinating case management and job search services, which were pro- 
vided by two different organizations, and this affected client flow through the job search ser- 
vices. Houston tried to use job development services provided by One-Stop staff or the ERA 
program but found that many of the openings required skills that participants did not have. 
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Individuals who did not find work by the end of the job search period generally participated 
in community service or a volunteer position at an employer. ERA case managers had a number of 
potential slots available, and they helped participants locate the one that best fit their interests. 
Community service positions generally required 20 hours or more of participation per week and 
continued until the individual found a job or left TANF. Staff reported that few individuals took 
community service positions, because they either found jobs or left TANF for other reasons. 

Participation in the ERA program was mandatory except for those who had a child 
younger than age 1, were ill or disabled, or were caring for a disabled family member. (Indi- 
viduals who were not mandated to participate were known as “exempt.”) 4 ERA case managers 
took the participation mandate seriously, by closely monitoring attendance and sanctioning in- 
dividuals if they did not attend program activities without good cause. For most of the study 
period, the sanction was a partial reduction of the family’s grant, although Texas adopted a full- 
family sanction in June 2003. Exempt individuals were eligible for all components of the ERA 
program and were strongly encouraged to participate — and it appears that many did so. 5 

In terms of preemployment services, the Choices programs across the sites were rela- 
tively strong and similar to the ERA programs. In addition, these services in Choices appear to 
have grown stronger over time, particularly after the waiver from federal participation require- 
ments ended. After this, Choices developed a stronger focus on immediate employment and 
made a more concerted effort to increase participation levels in program services in order to 
meet federal requirements. Although there was some variation across the sites, Choices typi- 
cally included the following preemployment services: 

• Participants were required to attend a workforce orientation on the Choices 
program in order to be certified for TANF. 

• A basic assessment was followed by four to six weeks of job search as the 
initial activity. Participants worked one-on-one with their case manager to 
find jobs and were required to make a certain number of employer contacts 
per week. 

• In some cases, a job readiness workshop (in addition to the one-on-one job 
search assistance) was provided, but not as routinely as in ERA. Choices also 
generally put less emphasis on longer-term career planning. 



4 As noted in Chapter 1, about 25 percent of those assigned to ERA were exempt from the participation re- 
quirement. 

5 Data from the ERA 12-Month Survey (see Chapter 3) show that mandatory and exempt individuals par- 
ticipated in employment services at comparable levels. 
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• Community service or a volunteer position was provided for individuals who 
did not find employment by the end of the job search period. 

• Like ERA, Choices was a mandatory program, and participants faced the 
same sanctions as under ERA if they did attend services as required. 

Postemployment Services: Retention and Advancement Case 

Management Services 

For employed participants, the goal of the ERA program was job retention and advance- 
ment in the labor market; both case management and financial incentives were provided to 
achieve this goal. As discussed above, individuals in the ERA program who found jobs received a 
four-month earnings disregard prior to receiving the monthly stipend that was part of the standard 
TANF program in Texas (and was also available to control group members in Choices). During 
the earnings disregard period, ERA’s case management services were as follows: 

• Except in Fort Worth, preemployment case managers (rather than retention 
and advancement staff) were generally required to have monthly contact with 
working participants during the earnings disregard period. In order to deter- 
mine continued eligibility for TANF and the disregard, case managers were 
required to verily the hours and wages of employment monthly, through 
these contacts, but they also used the opportunity to address any job-related 
issues. The contacts often but not always took place over the phone, and par- 
ticipants faxed any necessary paperwork to the case manager. 

• ERA participants typically met with postemployment staff once during the 
disregard period to encourage individuals to take advantage of the stipend af- 
ter the period ended and to review the requirements to receive it (particularly 
the hours of work required, the documentation needed, and the required 
monthly employment-related activities). 

Retention and advancement services were strongest once individuals started receiving 
the stipend. At this point, postemployment staff were expected to continue to make at least 
monthly contact with individuals on their caseload. During the initial stages of employment, 
staff focused on job retention and job-related problems while ensuring that individuals met the 
requirements to receive the stipend. However, as clients became stabilized in their jobs, staff 
also discussed advancement options — primarily in terms of the next job at the current em- 
ployer or a job change. 

In order to address both retention and advancement issues more effectively, the ERA 
programs in Corpus Christi and Fort Worth conducted many of their monthly meetings with 
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stipend recipients at the worksite — although Fort Worth did not begin employer site visits until 
later in the study period. Because it was difficult to schedule office visits when participants were 
working and because staff found in-person meetings more valuable than phone contacts, Corpus 
Christi initiated the practice of routinely meeting at the worksite. These meetings generally in- 
cluded both the employee and the employer in discussions of job-related issues, job perform- 
ance, and advancement options. (See Box 2.2.) 



Box 2.2 

Employer Site Visits in Corpus Christi 

In Corpus Christi, for most of the study period, postemployment staff conducted employer site 
visits at 2 weeks and at 1, 3, 6, 12, and 18 months after an individual found a job; staff later 
moved to the goal of monthly site visits whenever possible. During the employer site visit, pro- 
gram staff spoke to both the worker and the supervisor about job performance and any issues 
that might have arise on the job (such as attendance, punctuality, or relationships with cowork- 
ers). After the individual had been stabilized on the job, staff members discussed advancement 
with both the employee and the employer. ERA staff worked with participants on things that 
were needed in order to move into a better job, such obtaining more skills and training or taking 
greater initiative on the job. The employer was also asked what workers needed to do to ad- 
vance in their jobs and how often the employer evaluated workers and gave raises. 

ERA staff conducting employer site visits noted that not all employers were amenable to 
meeting with them about specific employees but that most were. To gain employers’ buy-in 
to the program, staff found that it was important to explain the goals of the ERA program, to 
emphasize the assistance that they could provide (improving job retention by addressing 
problems, assisting with staff development, and assisting in filling future vacancies), and to 
build a relationship with the employer. Staff also found that it was helpful to keep the meet- 
ings with employers “friendly and casual” and short (lasting less than five minutes). Staff re- 
ported that most employees were amenable to the site visits because it saved them a trip to 
the ERA office or trying to connect by phone. If an employer or employee did find the visits 
problematic, ERA staff arranged to meet with the employee during lunch hour or a break or 
at an offsite location. 



In Houston, due to administrative issues, ERA staff were generally unable to conduct 
employer site visits, and so they maintained contact with working participants through office 
visits and phone calls. In general, for much of the study period, the Houston program placed a 
higher priority on enrolling welfare recipients and on delivering preemployment services. This 
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site focused on developing more comprehensive retention and advancement services near the 
end of the study period. 

Despite the ERA program’s retention and advancement services and financial incen- 
tives, staff reported that job loss was more pervasive and occurred more quickly than they ex- 
pected. Although staff knew that job loss was a common problem for the ERA population, they 
were still surprised by its magnitude. Sometimes participants informed staff when they were 
going to lose a job or quit, but other times staff did not find out until attempting their monthly 
contact. Because of this, all the sites found that they needed to strengthen their reemployment 
services. Corpus Christi and Fort Worth had goals of reemploying ERA participants within two 
weeks of finding out about the job loss. This assistance was provided regardless of whether the 
individual was on TANF or not. Fort Worth also designated a case manager to address all cases 
who needed reemployment services. 

In Choices, the postemployment services consisted primarily of monthly contact with 
participants to monitor and verily employment status while they were receiving the four-month 
earned income disregard. Choices case managers generally did not maintain contact with indi- 
viduals once they left TANF and were no longer receiving the disregard. Choices also did not 
offer reemployment assistance unless an individual reapplied for TANF, and then it offered 
standard Choices preemployment services. 



Financial Incentives: Administration and Use of the 
Postemployment Stipend 

As discussed above, to encourage job retention and advancement, the Texas ERA pro- 
gram used an innovative approach: a payment of $200 per month for up to 12 months. This sti- 
pend was available after the earned income disregard period for individuals who worked 30 
hours per month or who worked part time in combination with attending school. The following 
sections discuss the marketing and use of the stipend, why some individuals did not use it, and 
the experiences of participants who did use the stipend. 

The Marketing and Use of the Stipend 

The Texas ERA program made a solid effort to market the stipend to participants. Par- 
ticularly in the early stages of the program, take-up of the stipend was lower than expected, and 
all three sites increased their marketing efforts over the course of the study. Staff had initially 
thought that the stipend would essentially “sell itself’ — given its relatively generous level and 
minimal requirements (particularly compared with TANF). However, staff across the sites 
eventually recognized the need to adopt a multifaceted marketing approach that focused primar- 
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ily on the stipend as a way to increase participants’ income but that also increased the incentive 
for participants to find and keep jobs. 

Corpus Christi moved most quickly to develop a strong marketing effort, and many of 
its strategies were adopted by Fort Worth and Houston. These marketing strategies included: 

• Telling participants about the stipend and strongly encouraging them to take 
advantage of it at all junctures of the program — immediately after partici- 
pants were assigned to the program, at their assessment, during job search ac- 
tivities, when they found a job, and when they had finished the earnings dis- 
regard period. 

• Displaying posters and fliers throughout the ERA offices to inform partici- 
pants about this benefit. 

• Developing strategies to make the stipend sound more straightforward and 
appealing. For example, staff started calling the financial incentive an “em- 
ployment bonus” — because many clients did not understand what a stipend 
is (some thought it was a loan that had to be paid back). Staff also started 
emphasizing that participants could receive a total of $2,400 — because the 
larger amount obviously had greater appeal. 

• Creating posters and videotapes to tell the “success stories” of participants 
who had received stipends and how they had used the resources. 

While the sites made a good-faith effort to market the ERA stipend, it appears that their 
efforts were not as strong as those made in other programs that provided a financial incentive to 
encourage work. (See Box 1.1 in Chapter 1.) For example, the Canadian Self-Sufficiency Pro- 
ject (SSP) provided an hour-long individual orientation that focused exclusively on the pro- 
gram’s earnings supplement and included brochures and illustrations of various financial sce- 
narios to help the client understand the incentive. In contrast, the Texas ERA program often 
provided information about the stipend along with a range of other information about different 
program rules and requirements. Although the stipend information was repeated consistently 
throughout the ERA program, because it was delivered among many other program features, it 
may not have been as effective as a message that focused exclusively on the financial incentive. 
The marketing of the ERA stipend may have been particularly challenging, given that receipt of 
the stipend could seem far in the future to some participants and was contingent on achieving 
several outcomes in addition to becoming employed (discussed further below). 
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Steps Required to Receive a Stipend 

After ERA participants had received the entire four-month earned income disregard, 
several steps were required in every month that they wanted to receive the $200 postemploy- 
ment stipend: 

• Participants had to work the required number of hours, which was generally 
30 hours per week in the given month. Although ERA allowed individuals to 
receive a stipend if they were going to school and working part time, this was 
not strongly promoted — in part because of TANF’s emphasis on employ- 
ment (preferably full time). Staff also reported that participants were not in- 
terested in this option and preferred to work full time. Staff did encourage 
ERA participants to pursue additional education and training — primarily, to 
get a GED certificate and some job training — but to do that in addition to 
working full time. 

• Participants had to submit documentation of the hours they worked. This was 
usually a paycheck stub, which could be faxed to the ERA office or provided 
in person to the case manager. 

• Finally, participants had to attend some type of employment-related activity 
each month. The Texas sites implemented this requirement in different ways 
but generally were flexible in what they allowed to count toward the monthly 
participation requirement — including training at work, education and training 
programs, various support groups, or other activities in the community. Each 
site also operated its own workshops or support groups that participants could 
attend to meet the requirement, and each also handed out a monthly calendar of 
both ERA-operated support groups and other community events that could be 
used to fulfill this requirement. In Corpus Christi and later in Fort Worth, staff 
distributed stipend checks at a monthly workshop, which allowed participants 
to get their check and meet the requirement at the same time — a popular ap- 
proach. Otherwise, stipend checks were mailed to participants. 

Initially, all the sites experienced some difficulty issuing the stipend checks in a timely 
manner, particularly because they did not have experience disbursing funds on a monthly basis. 
There were also some issues regarding how to verify that participants had worked enough hours 
to qualify for the stipend. Corpus Christi at first required a work supervisor’s signature, but par- 
ticipants sometimes had difficulty obtaining this or did not want to ask for it. Such issues were 
resolved relatively quickly, however, and most stipend checks were issued on time. 
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The Use of the Stipend 

To understand the Texas program’s success in marketing and administering the 
postemployment stipend, this section examines the proportion of individuals in the ERA pro- 
gram group who received a stipend and the frequency and duration of stipend use. 

Stipend Receipt Rates for the Program Group 

To examine stipend receipt rates, administrative data were collected for all individuals 
who were assigned to the ERA program. This sample includes many individuals who did not 
quality for the stipend because they did not find jobs, did not work enough hours, or did not 
work longer than the four-month earned income disregard period. As Table 2.2 shows, stipend 
receipt rates were highest in Corpus Christi, where 30 percent of all ERA group members re- 
ceived a stipend (including those who found jobs as well as those who did not) through the du- 
ration of the program. The receipt rate was lower in Fort Worth and Houston, where about 20 
percent of ERA group members ever received a stipend. Figure 2.2 shows the proportion of the 
ERA group in Corpus Christi who received the stipend in each quarter as well as when indi- 
viduals received their first stipend. Stipend receipt rates increased relatively quickly through the 
first and second year before leveling off in the third year. (Appendix E presents results for Fort 
Worth and Houston.) 

Stipend Receipt Rates Among Key Subgroups 

Figure 2.3 shows stipend receipt rates among key subgroups. (Chapter 4 also analyzes 
these subgroups in terms of program impacts.) The figure shows that stipend receipt was highest 
among those who had a strong employment history prior to random assignment. In Corpus 
Christi, for example, 40 percent of those who earned more than $5,000 in the year prior to ran- 
dom assignment received a stipend, compared with only 30 percent of the full sample. This 
subgroup may be more likely to find jobs and benefit from the postemployment services pro- 
vided through ERA. In contrast, only 26 percent of those who were unemployed in the year 
prior to random assigmnent received a stipend. Other subgroups — such as those based on wel- 
fare history or high school graduation — did not show any differences in stipend receipt rates 
(not shown in the figure). 

Stipend Receipt Rates Among Those Who Were Eligible 

Individuals who found jobs and received the entire four-month earned income disregard 
generally became eligible for the stipend — although they may have not received it if they did 
not work enough hours, submit the required documentation, or attend the required activity in a 
given month. Clearly, compared with the whole sample, a higher proportion of those who be- 
came eligible for the stipend did receive one. However, due to data limitations, it is not possible 
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The Employment Retention and Advancement Project 
Table 2.2 

Receipt Rates, Timing, and Duration of Stipends in the ERA Program 1 * 

Texas 



Outcome 


Corpus 

Christi 


Fort 

Worth 


Houston 


Amone all Droeram group members 


Ever received a stipend (%) 


30.0 


21.7 


20.4 


Sample size (total = 2,857) 


988 


821 


1,048 


Among those receiving a stipend 


Average number of months to first stipend 


16.8 


14.4 


17.1 


Number of months to first stipend 


Less than 6 


15.4 


20.7 


9.8 


6 to 12 


26.2 


30.7 


28.5 


13 to 24 


35.5 


33.6 


38.4 


Over 24 


22.9 


15.0 


23.3 


Total number of stipends (%) 


1 


11.1 


5.6 


17.8 


2 


4.7 


6.7 


10.3 


3 


3.7 


6.2 


12.1 


4-6 


17.9 


15.2 


24.8 


7-10 


13.9 


16.9 


25.7 


1 1 or more 


48.6 


49.4 


9.3 


Average amount received ($) 


1,631 


1,681 


1,247 


Average number of months stipend received 


8.2 


8.4 


5.2 


Sample size (total = 688) 


296 


178 


214 



SOURCE: MDRC calculations from ERA program tracking data. 

NOTE: ‘‘Among all ERA group members randomly assigned October 2000 through January 
2003. 
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The Employment Retention and Advancement Project 
Figure 2.3 

Percentage Receiving the ERA Stipend Among the Full Sample and 
Key Subgroups Defined by Pre-Random Assignment Status 
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to estimate who was eligible for a stipend. To provide a rough idea of the receipt rate among 
those who were eligible, Figure 2.4 estimates the proportion of people who were likely to have 
gone through the steps involved in receiving a stipend in Corpus Christi. (Appendix E presents 
estimates for the other two sites in Texas.) Figure 2.4 shows that: 

• Out of every 1 00 people randomly assigned to ERA, 84 met the first criterion 
for receiving a stipend: They were detennined eligible for the TANF pro- 
gram and received benefits. As noted above, individuals were not eligible for 
a stipend unless they received TANF cash assistance. 

• Of those 84 people, 70 became employed at some point during the follow-up 
period. However, to receive a stipend, an individual must have worked be- 
yond the four-month earned income disregard period. Because participants 
could receive the disregard only once in a 12-month period (and the disre- 
gard did not have to be received in consecutive months), the eligible popula- 
tion was estimated as those employed individuals who worked in two or 
more quarters within a year and who earned over $2,400 in these two quar- 
ters, based on unemployment insurance (UI) data. 6 This was done to elimi- 
nate those who worked very little over the two quarters and who would not 
have received the disregard for the full period. Figure 2.4 shows that an esti- 
mated 55 of the 70 employed participants in Corpus Christi appear to have 
worked enough to receive the entire four-month earnings disregard. 7 

• Of the 55 who worked at least four months, 30 individuals (or 55 percent) re- 
ceived a stipend. While this is not a precise estimate of the stipend receipt 
rate among those who were eligible, it shows that Corpus Christi, in particu- 
lar, was effective in encouraging the take-up of the incentive for at least a 
moderate proportion of the eligible population. In Fort Worth and Houston, 
approximately 40 percent of those who worked longer than four months re- 
ceived a stipend. (See Appendix E.) 

These take-up rates are lower than were found in other studies of programs using financial 
incentives. In SSP, for example, a similar proportion — approximately 35 percent of the program 



6 For the earnings threshold, it was assumed that an individual worked 20 hours per week at $7 per hour for 
four months. 

7 Because the earnings threshold that was used to determine who would be eligible for a stipend is an esti- 
mate, a sensitivity analysis was conducted to determine how the results would change using different earnings 
thresholds. Overall, this analysis did not find large differences in terms of the number who would be eligible 
for the stipend. 
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The Employment Retention and Advancement Project 

Figure 2.4 

Estimated Eligibility for and Use of the ERA Stipend Among 100 TANF Applicants and 

Recipients in Corpus Christi* 




SOURCES: MDRC calculations from UI administrative records from the State of Texas and ERA program tracking 
data. 

NOTES: “Among all ERA group members randomly assigned from October 2000 through January 2003. 

b The proportion of individuals who worked for four months and thereby completed the earnings disregard period 
was estimated using UI records. Individuals were determined to have worked for four months if they were employed 
for two consecutive quarters within a year and had total earnings of more than $2,400 during this period (this is the 
equivalent of working 20 hours per week for four months at $7 per hour). 



37 











group — used the financial incentive offered in that program. 8 However, in contrast, almost all the 
individuals who were eligible to receive the SSP incentive (because they worked full time) did so. 
In the Minnesota Family Investment Program (MFIP), because the financial incentive was pro- 
vided in the welfare grant in the form of an earnings disregard, all the individuals who went to 
work automatically received the payment for as long as they remained on assistance. 

The Timing of Stipend Receipt 

As Table 2.2 shows, it was a relatively long period of time before individuals received 
their first stipend — about 17 months, on average (but less time in Fort Worth). In part this re- 
flects that individuals were receiving their first stipend well into the third and fourth years of the 
follow-up period (Figure 2.2). In Corpus Christi, about 40 percent receive their first stipend in 
the first year; one-third did so in the second year; and one-fifth waited until the third year. As 
discussed above, this partly reflects the number of steps required before receiving a stipend, in- 
cluding participating in preemployment services, finding a job, and completing the earned in- 
come disregard period. It also could reflect the ongoing outreach efforts of staff to bring back 
into the program those who had left or had never participated. 

The Duration of Stipend Receipt 

Particularly in Corpus Christi and Fort Worth, many of those who did receive a stipend 
continued to receive it. Almost half of stipend recipients in those two sites received eleven or 
more monthly payments (Table 2.2). In Houston, nearly two-thirds of those who received a sti- 
pend received six or fewer payments, and only 9 percent received eleven or more. Similarly, 
among recipients in Corpus Christi and Fort Worth, the average number of months of stipend 
receipt was eight, compared with an average of five months in Houston. Among those who re- 
ceived stipends, the total payments averaged $1,631 in Corpus Christi; $1,681 in Fort Worth; 
and $1 ,247 in Houston. 

To increase the stipend receipt rate, all three Texas sites expended considerable effort try- 
ing to locate individuals who had previously been in the ERA program but were not currently par- 
ticipating or who had never participated. Once these people were located, staff engaged them in 
job search services (if not employed) or encouraged them to take advantage of the stipend (if em- 
ployed). Based on interviews with staff, these efforts appear to have had some, but limited, effects 
on increasing the receipt of stipends. This was a greater concern in Houston and Fort Worth, 
where the programs lost touch with some participants early on because of startup issues. Although 
these sites had organized outreach efforts to contact such individuals, staff were unable to get 



8 Michalopoulos et al. (2000). 
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many of them back into the ERA program. Many fonner participants were difficult to locate or 
could not be contacted, and many nonparticipants were still not interested in the program. 

Reasons for Not Using the Stipend 

While many ERA participants did use the monthly stipend, some working individuals 
clearly did not use it. There are several reasons why this could have occurred: The individual 
may not have met specific requirements, such as working enough hours in a particular month or 
attending the monthly activity; the individual may not have known about or understood the sti- 
pend; or the individual may not have wanted the stipend. Overall, it appears that different peo- 
ple did not use the stipend for different reasons, with no single reason dominating. 

As one way of examining this issue, MDRC staff held telephone discussions with a 
small number of individuals (three in Corpus Christi and five in Houston) who had not received 
the stipend even though they appear to have been eligible for it, based on a review of the pro- 
grams’ employment records (that is, they had been employed for four or more consecutive 
months ). 9 Given the small sample size, these individuals are not representative of all those who 
did not receive stipends when they may have been eligible, but their views are suggestive of 
hypotheses. Most of the respondents said they knew about the availability of the stipend, but 
two did not. Of the six who knew about it, a few said that they were not receiving the stipend 
because they found the monthly workshop requirement too burdensome; for example, one 
mother did not have anyone to take care of her children during the workshop’s scheduled time. 
Other reasons for not using the stipend include not working enough hours, thinking that they 
would not be eligible for the stipend (and not being in communication with ERA staff), and not 
being sure why they were not receiving the stipend. 

Several of these telephone respondents reported that they had limited interaction with 
ERA staff and did not use the services very much, although some had found the services helpful 
in the past. Several were no longer working (for a variety of reasons, including quitting, being 
fired, and being laid off), but most were not interacting with ERA staff to become reemployed. 
Most said they did not know that they could receive the stipend by attending school and work- 
ing part time, but most were not interested in this option because of time constraints. 

To understand why some ERA participants did not use the stipend, MDRC also con- 
ducted interviews with ERA staff, who again offered a range of reasons. First, some clients lost 
a job before the earned income disregard period ended. Several staff reported that this some- 
times was beyond the control of the participant but that, in other cases, participants had a “fear” 



9 Repeated attempts to contact a much greater number of individuals about their reasons for not using the 
stipend were unsuccessful. The response rate for this effort was 23 percent. 
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of leaving TANF or they left the job for other reasons. In some cases, participants could not 
meet the stipend requirements; they either did not attend the monthly job-related activity or did 
not work enough hours in the month. Staff also reported that some participants felt a stigma as- 
sociated with the stipend — much like being on cash assistance — and were not interested in 
continuing involvement with a “government program” after leaving TANF. Finally, staff re- 
ported that some clients were concerned about losing TANF’s child care assistance. Even 
though ERA participants were eligible for transitional child care subsidies when they left 
TANF, some participants were concerned because ERA had a somewhat higher child care co- 
payment (although the stipend could be used to cover it), and this caused some to quit their job 
before the disregard period ended. 

The Experiences of Individuals Who Received a Stipend 

To better understand how the ERA participants who received a stipend used these 
funds, MDRC also held telephone discussions with several stipend recipients (five in Corpus 
Christi and seven in Houston). Again, because of the small sample size, the responses of these 
individuals are simply suggestive. Most respondents had received the stipend for at least several 
months, and three had received it for all 12 months. 

Overall, these individuals found the stipend to be very helpful in covering household and 
work-related expenses. Most used the stipend to pay for rent, other household bills, food, child 
care or transportation. A few tried to save some of the funds for an emergency, although this was 
not as common. A few also used the money to buy something for their children. Although all re- 
spondents found the stipend useful, they consistently stated that the availability of the stipend did 
not affect their decision to take a job or stay in it — they would have made the same decisions 
even without this monthly incentive. A few individuals had lost their job at the time of the inter- 
view, primarily because they had been laid off or because the job had been temporary. 

Most of the individuals in this group did not find the requirement to attend a monthly 
workshop burdensome, and many were enthusiastic about the sessions: “I love them”; the in- 
structor makes it “fun for us”; they provide “great information”; and they “really opened my 
eyes.” Several said that having a Saturday session to which they could bring their children made 
it easier to meet this requirement. But a few individuals did find the sessions difficult to get to, 
particularly given their work schedules. Three respondents (across both sites) reported experi- 
encing at least some delay in receiving a stipend check. 

Most of the individuals in this group spoke positively about their interactions with ERA 
staff. The type of assistance they received from staff varied but included help with transportation, 
child care, housing, work supplies, job search, and finding a better job. While one person was 
combining part-time work and attending a GED program in order to receive the stipend, most in- 
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dividuals said that they did not know they could also receive the stipend if they attended school 
part time while working. But these respondents also said that they would not be interested in this 
option at the moment because they did not have the time and had too many other responsibilities. 



How Did ERA Staff Spend Their Time? 

MDRC administered a “time study” in all the ERA sites to better understand the prac- 
tices of program staff and what it takes to operate a program like ERA. The study captured de- 
tailed infonnation on the nature of interactions between ERA staff and clients and on the topics 
covered in their interactions. It also collected infonnation on how ERA staff typically spent 
their time each day. In Texas, the time study was administered over a two-week period in July 
2003 in Corpus Christi and Houston and in February 2004 in Fort Worth — which is toward the 
end of the follow-up period for this report. During this time, all staff who worked directly with 
ERA participants — including pre- and postemployment case managers and staff who worked 
with participants to find jobs — recorded their activities each day, using fonns designed by 
MDRC. At that time, very few new cases were being assigned to staff, although they continued 
to work with ongoing cases. 

When the time study was administered, the caseloads of individual ERA staff ranged 
from 61 in Corpus Christi to 76 in Houston. Reflecting that the time study was conducted when 
few new clients were enrolling in the program, over half the caseload in each site were working 
(over 80 percent in Fort Worth were working). Caseload sizes varied somewhat across staff and 
types of staff but were generally in the range of other ERA programs. 

Figure 2.5 shows that when the time study was administered, staff spent about one- 
quarter of their time in contact with clients (ranging from 22 percent of the time in Corpus 
Christi to 29 percent in Houston). This is typical across the ERA sites. Staff spent about equal 
amounts of time with working and nonworking clients — except in Houston, where almost 
twice as much time was spent with working clients. Although activities varied somewhat across 
the sites, staff spent the remainder of their time doing administrative duties, participating in 
meetings, completing notes in the Management Information System (MIS), or attempting to 
contact participants (outreach). Notably, staff in Corpus Christi spent 8 percent of their time 
interacting with employers — one of the highest rates among all the ERA sites. 

As shown in Table 2.3, the number of contacts per day that ERA staff had with program 
participants ranged from five in Corpus Christi to eight in Fort Worth. The average contact 
lasted about 18 minutes in Corpus Christi and Houston and was somewhat shorter in Fort 
Worth. Overall, staff in Fort Worth had more but shorter contacts than staff in the other two 
sites. About an equal amount of time was spent with working and nonworking clients — except 
in Houston, where contacts with nonworking clients were longer. 
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The Employment Retention and Advancement Project 
Figure 2.5 

Summary of How Texas ERA 
Case Managers Typically Spend Their Time 



Client contact 
(22% of all 
time) 



Corpus Christi 



Working clients 



Other activities 
(78.5% of all time) 



Nonworking clients 
11 % 




Administrative duties 
13% 



Staff meetings 
6% 



Other 

19% 



Outreach 

14% 



Completing counselor 
notes 
14% 



Traveling 

5% 



Interacting with 
employers 
8 % 



Client contact 
(29% of all time) 



Houston 




Working clients 
22 % 



Administrative duties 
10 % 



Nonworking clients 
12 % 

Other 

4% ~ 

Monitoring client 
participation in services 
4% 




Other activites 
(71% of all time) 



Staff meetings 
10 % 



Outreach 

9% 



Completing counselor 
notes 
9% 



TANF participation 
7% 



Traveling Job development 



6% 



(continued) 
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Figure 2.5 (continued) 



Client contact 
(28% of all time) 



Fort Worth 



Working clients 
12% 



Nonworking clients 
16% 



Administrative duties 
16% 



Other activities 
(71% of all time) 




Staff meetings 
7% 



Observing or assisting 
with specific program 
activities 
11% 



Other 

23% 



Completing counselor 
notes 
9% 



Monitoring client 
participation in 
services 6% 



SOURCE: MDRC calculations for the ERA time study. 



Table 2.4 shows the type and location of all client contacts throughout the period stud- 
ied. In Corpus Christi, contacts were split evenly between those that took place in person and 
those that did not, whereas more than 60 percent of contacts in the other sites were not in per- 
son. In all three sites, one-third of contacts occurred in the office. Corpus Christi was unique 
because it also conducted in-person visits to clients’ employers and homes (9 percent and 6 per- 
cent of all contacts, respectively), whereas most contacts in the other sites were by phone. 10 In 
Corpus Christi and Fort Worth, staff initiated most of the contacts, but almost three-quarters of 
the contacts in Houston were initiated by clients. 

Table 2.5 shows the percentage of client contacts that included discussion of various 
topics. Reflecting the emphasis on marketing the stipend, the most common topic/activity 
across all three sites was discussing issues related to the stipend (included in 23 percent to 44 
percent of discussions). “General check-ins” were common in Corpus Christi and Houston; 
Houston had a large proportion of contacts addressing reemployment; and Corpus Christi and 
Fort Worth had a large proportion of contacts that included a discussion of participation and 



"'Although Fort Worth staff also conducted employer site visits during the period when the time study was 
conducted, they did not do any visits during the two weeks when the time study was administered. 
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The Employment Retention and Advancement Project 
Table 2.3 

Extent of Contact Between ERA Case Managers and Clients 

Texas 





Corpus 

Christi 


Fort 

Worth 


Houston 


Percentage of work time spent in contact with 


Any client 


21.5 


28.5 


29.2 


Working clients 


11.3 


12.0 


18.5 


Nonworking clients 


10.2 


16.5 


10.7 


Work experience clients 


NA 


NA 


NA 


Average number of client contacts per day per case manager 


Any client 


4.9 


8.1 


5.7 


Working clients 


2.6 


4.2 


3.8 


Nonworking clients 


2.3 


3.9 


1.8 


Work experience clients 


NA 


NA 


NA 


Average number of minutes per contact with 


Any client 


18.6 


14.8 


18.3 


Working clients 


15.6 


13.1 


16.3 


Nonworking clients 


16.2 


16.8 


23.5 


Work experience clients 


NA 


NA 


NA 


Number of case managers time-studied 


9 


6 


7 



SOURCE: MDRC calculations from the ERA time study. 



NOTE: NA = not applicable. 



sanctioning issues. Discussions of career goals and advancement occurred in 10 percent to 16 
percent of the client contacts. 



Variations in Implementation Across Sites 

As the above discussion indicates, there was significant variation in how the three 
Texas programs implemented the ERA model. Overall, implementation was smoothest in Cor- 
pus Christi — the site that moved most quickly to establish a staffing structure and program 
services reflecting all aspects of the model. This included preemployment services with some 
focus on longer-term career planning; intensive postemployment case management, including 
regular employer site visits; and strong marketing of the stipend. Many of the practices in Cor- 
pus Christi — such as the specialized staffing arrangements, practices for marketing and dis- 
tributing the stipend, and employer site visits for individuals receiving the stipend — were even- 
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The Employment Rentention and Advancement Project 
Table 2.4 

Description of Contact Between ERA Case Managers and Clients 

Texas 





Corpus 

Christi 


Fort 

Worth Houston 


Percentage of all client contacts that were: 








In person 


51.0 


37.9 


36.4 


Office visit 


36.5 


34.8 


33.1 


Home visit 


6.0 


1.2 


0.9 


Employer visit 


8.6 


0.0 


1.1 


Elsewhere 


0.0 


1.9 


1.3 


Not in person 


49.0 


62.1 


63.6 


Phone contact 


40.5 


61.8 


61.8 


Written contact 


8.5 


0.0 


1.8 


Other type of contact 


0.0 


0.3 


0.0 


Percentage of all client contacts, over a two-week period, that were initiated by: 








Staff member 


59.2 


57.1 


26.3 


Client 


40.5 


42.5 


73.7 


Another person 


0.3 


0.3 


0.0 


Number of case managers time-studied 


9 


6 


7 



SOURCE: MDRC calculations from the ERA time study. 

tually adopted by the other two sites. Corpus Christi was also the only site to implement a com- 
prehensive performance measurement system for staff. 

The Fort Worth program struggled for a good portion of the study period, particularly in 
designing effective preemployment services. Initially, the program strongly emphasized as- 
sessment and removal of employment barriers, which limited the extent to which participants 
moved into employment and postemployment services. Coordination issues between the two 
agencies that were involved also affected client flow through the program’s components. When 
a new manager was hired in 2002, however, the Fort Worth program made significant im- 
provements, including more structured job search services and stronger postemployment case 
management services, along with regular employer site visits. During the early period, though, 
the program lost track of many individuals and could not locate many of them after things im- 
proved (although attempts were made). Thus, the stronger set of services was provided to only a 
portion of the participants who were assigned to the ERA group. 
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The Employment Retention and Advancement Project 
Table 2.5 

Topics Covered During Contact Between ERA Case Managers and Clients 

Texas 





Corpus 

Christi 


Fort 

Worth 


Houston 


Percentage of all client contacts that included the following topics: 3 








Initial client engagement 


16.8 


15.9 


7.6 


Supportive service eligibility and issues 


15.1 


12.4 


12.5 


General check-in 


27.7 


5.8 


25.1 


Screening/assessment 


10.3 


2.7 


9.3 


Address on-the-job issues/problems 


8.3 


0.2 


13.4 


Address personal or family issues 


7.2 


5.1 


17.3 


Explore specific employment and training options 


10.3 


2.7 


7.3 


Discuss career goals and advancement 


16.2 


9.6 


13.8 


Assist with reemployment 


11.5 


7.3 


20.6 


Discuss issues related to financial incentives or stipends 


31.3 


23.0 


43.7 


Schedule/refer for work experience position 13 


NA 


NA 


NA 


Enrollment in government assistance and ongoing eligiblity issues 


0.0 


9.2 


1.1 


Assistance with the EITC 


0.0 


1.2 


0.0 


Participation/sanction issues 


28.6 


23.8 


8.5 


Schedule/refer for screening/assessment 


1.2 


1.4 


2.1 


Schedule/refer for job search or other employment services 


6.2 


2.7 


5.1 


Schedule/refer for education or training 


0.3 


3.3 


1.6 


Schedule/refer for services to address special or personal issues 


1.1 


2.0 


3.7 


Provide job leads or referrals' 3 


NA 


NA 


NA 


Number of case managers time-studied 


9 


6 


7 



SOURCE: MDRC calculations from the ERA time study. 

NOTES: NA = not applicable. 

“Percentages total over 100 percent, since more than one topic could be recorded for each client 
contact. 

b This measure was not included in the time-study instrument used in Texas. 
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The Houston program moved most slowly to get ERA’s retention and advancement 
services off the ground. For much of the study period, this program placed a higher priority on 
outreach and recruitment of new clients and developing preemployment services; postemploy- 
ment services were not fully developed until very late in the study. Local administrative issues 
also made it difficult for this program to implement certain features that the staff felt would 
have been beneficial — particularly, employer site visits for working individuals and job devel- 
opment staff that would be dedicated to ERA participants. 
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Chapter 3 



The Effects of the Texas ERA Program 
on Message and Service Receipt 



Chapter 2 describes the Employment Retention and Advancement (ERA) programs that 
were implemented and studied in three Texas sites: Corpus Christi, Fort Worth, and Houston. 1 
This chapter provides infomiation about participation in the program and other similar services, 
focusing on the differences between the experiences of individuals in the ERA group and those in 
the control group. Examining these differences is central to understanding the outcomes on em- 
ployment, public assistance, and income that are presented in Chapter 4. As noted previously, con- 
trol group members were not eligible for ERA services but could receive services through 
Choices, the state’s standard welfare-to-work program for recipients of Temporary Assistance for 
Needy Families (TANF), as well as through other programs and agencies in their area. 

This chapter relies primarily on data from the ERA 12-Month Survey, which was ad- 
ministered to a subset of ERA and control group members in Texas about 12 months after they 
entered the study. Chapter 1 notes that the Fort Worth program administered the survey to a 
later cohort of sample members, who were thus more likely to receive the stronger program that 
developed later in the study period. 2 Across the three sites, a total of 775 sample members re- 
sponded to the survey. Respondents were asked a series of questions about their contact with 
case managers and similar staff and about their participation in employment-related activities. 
This chapter does not discuss receipt of the postemployment stipend. As noted in Chapter 2, 
program administrative records show that a moderate portion of ERA group members received 
this benefit, which was not available to control group members and thus represented a key dif- 
ference in treatment between the two groups. 



Key Findings 

According to survey respondents, only the ERA programs in Corpus Christi and Fort 
Worth produced statistically significant increases in the proportion of ERA clients who received 
help with retention and advancement services. These services included help finding a better job 
while working, enrolling in life skills classes while working, and career assessment. Despite the 



'Although the Texas program operated in various cities that are called “sites” in this report, Texas counts 
as a single ERA site. 

2 The survey sample was selected from adults in single-parent families who spoke either English or Span- 
ish and who were randomly assigned from January through June 2002 in Corpus C hristi and Houston and from 
September through December 2002 in Fort Worth. 
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program’s impacts, the overall level of receipt of retention and advancement services is some- 
what lower than expected: Only about 40 percent of working survey respondents from the ERA 
program reported receiving assistance in this area. This could possibly reflect that — because 
some individuals did not receive the stipend and more intensive retention and advancement ser- 
vices until they had spent more than a year in the program (see Chapter 2) — this activity was 
not fully captured by the 12-month survey. 

In Corpus Christi and Fort Worth, ERA group members were more likely than control 
group members to have had contact with a case manager or employment program during the 
year after entering the program. The differences between the two groups are not large, however; 
this indicates that control group members in Choices were assigned to a relatively strong wel- 
fare-to-work program that included case management services. 

Reflecting Corpus Christi’s emphasis on employer site visits, staff contacts with ERA 
group members in that program only were more likely to occur at the workplace than were con- 
tacts with control group members, and staff were more likely to talk with an ERA respondent’s 
employer. However, the overall proportion of respondents in the ERA group who reported staff 
contacts at or with their employer is low. This is not surprising, given that many ERA group 
members did not reach the stage in the program where they were receiving the stipend and this 
type of postemployment service, as discussed in Chapter 2. Again, this activity may not be fully 
captured by the 12-month survey because stipend receipt (and, therefore, employer visits) oc- 
curred after more than a year in the program for some individuals. 

Overall, participation rates in employment-related services are relatively high in all 
three sites for both the ERA group and the control group, and there were few other differences 
in the types of services that each group received. Again, this shows at least in part the relative 
strength of the Choices program, particularly in terms of preemployment services. 



The Intensity and Nature of Contacts Between Clients and Staff 

As discussed in Chapter 2, a key element of the Texas ERA program was individual- 
ized case management services at both the pre- and the postemployment stage of the program. 
Staff provided one-on-one assistance to help participants find jobs, arrange support services, 
and address job-related issues after participants found jobs. A key issue, then, in assessing the 
strength of the program “treatment,” is the intensity and nature of contacts between staff and 
clients. The ERA 12-Month Survey asked a series of questions intended to capture the nature of 
contact between respondents and program staff and other staff from employment and social ser- 
vice agencies. Given the nebulous nature of “case management,” it was a challenge to design 
these questions. (See Box 3.1.) 
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Box 3.1 



Measuring Participation in ERA 

In order to interpret the results of a random assignment evaluation, it is critical to understand 
the “dose” of services that each research group receives. In many studies, this is relatively 
straightforward because the “treatment” is easy to measure (for example, number of hours of 
training or the dollar value of incentive payments). In contrast, in many of the ERA pro- 
grams, including Texas’s, services are mostly delivered in one-on-one interactions, during 
which staff advise, coach, or counsel participants. This type of service is inherently difficult 
to measure. In addition, to accurately measure a program’s impact on service receipt, it is 
important to collect data in the same way for both the ERA group and the control group. In 
practice, this means that survey questions cannot refer to the ERA program in particular but, 
instead, must ask in general about the kinds of services that ERA provided. 

MDRC sought to measure service receipt in three main ways, using the ERA 12-Month Sur- 
vey. Each approach has both strengths and limitations, and each contributes to the overall 
analysis: 

• First, the survey asked whether respondents participated in “traditional” employment- 
related services, such as job search workshops and training classes, and how many 
weeks they participated (see Table 3.3). These services are relatively easy to measure, 
but they are not the heart of most ERA programs. 

• Second, the survey asked how frequently respondents had contact with staff from em- 
ployment or social service agencies and where these contacts took place (see Table 3.1). 
These questions are more central to the ERA programs, but it is difficult to determine to 
which types of staff respondents were referring to. For example, contact with a worker 
who determines food stamps eligibility is likely to be quite different from contact with 
an ERA case manager. Moreover, it may be difficult for respondents to recall the num- 
ber of such contacts over a one-year period. 

• Third, the survey asked whether respondents received assistance in a variety of specific 
areas, some of which — such as “finding a better job while working” — are central to 
ERA (see Table 3.2). These questions are fairly straightforward, but they do not provide 
any information about the amount of service that was received in each area. 



To estimate contact between sample members (both ERA and control group members) 
and program staff or other organizations or agencies that individuals may work with to find or 
keep jobs, the analysis combined two survey questions into a single measure. One question asked 
whether the respondent had had contact with “programs or organizations that help people find and 
keep jobs.” The other question was introduced with a sentence that referred to “agency staff [who] 
help people find and keep jobs,” but it asked whether the respondent had had “contact with a case 
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manager or a staff person from an employment, welfare, or other agency.” In this chapter, the 
combined measure is referred to as “contacts with a case manager or employment program.” 

Table 3.1 presents the program’s impacts on contacts with staff during the year following 
random assignment. Within each site’s set of three columns, the first column presents outcomes 
for the ERA group; the second column presents outcomes for the control group; and the third col- 
umn presents the difference, or impact, between the two groups. Because random assignment en- 
sures that there are no systematic differences between the ERA and control groups when they en- 
ter the study, any differences in the groups’ outcomes that emerge over time can be attributed to 
the program intervention. Tests of statistical significance were perfomied on all impacts presented 
in this report, to determine whether an impact can confidently be attributed to the program. An 
impact is considered statistically significant at the 10 percent level if there is less than a 10 percent 
chance that the estimated difference could have stemmed from a program that had no real effect. 
Statistical significance is also presented at the 5 percent and the 1 percent levels. Except where 
otherwise noted, only statistically significant impacts are discussed in the text. 

As Table 3.1 shows, in Corpus Christi and Fort Worth (but not in Houston), the ERA 
program resulted in significantly more contacts with case managers after enrollment in the 
study. In these two sites, over 60 percent of the ERA group reported that they had contact with a 
case manager or employment program since they entered the study, while about 45 percent of 
the control group did. Although this difference, or impact, of 15.0 percentage points in Corpus 
Christi and 17.4 percentage points in Fort Worth is statistically significant, it is relatively small. 
Moreover, the overall level of contacts with case managers is relatively low. 

These results also show that the control group was assigned to a relatively strong wel- 
fare-to-work program, Choices, which also included case management services. In Houston, as 
in the other sites, roughly 60 percent of ERA sample members had contact with ERA staff; 
however, unlike in the other sites, a similar proportion of control group members in Houston did 
so as well. There was no impact on the level of contact with program staff in Houston, which 
appears to reflect particularly strong case management in that site’s Choices program. 

It is worth noting that the survey likely undercounted contacts with program staff for 
both research groups. The survey was administered a year after random assigmnent, and if sam- 
ple members had contact with ERA or another program early in that year or if the contact was 
not very intensive, respondents might not have remembered it. But since the survey captured 
contact that the respondents remembered, it is not unreasonable to assume that it captured most 
of the contact that mattered to them and was likely to affect their outcomes. 

Table 3.1 presents some details about sample members’ contact with program staff. 
Fort Worth was the only site to have a statistically significant impact on the average number of 
contacts with program staff, whether in person or by phone. ERA sample members in Corpus 
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Table 3.1 
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Table 3.1 (continued) 
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Christi and Fort Worth were also more likely than their control group counterparts to have spo- 
ken with a case manager in the four weeks before the survey. 

In Corpus Christi and Fort Worth, the largest increase was in the number of contacts 
that occurred at the case manager’s office. Notably, however — reflecting the emphasis on em- 
ployer visits in Corpus Christi — staff contacts with ERA group members were more likely to 
occur at the workplace than were contacts with control group members, and ERA staff were 
more likely to talk with an ERA respondent’s employer “once or twice” or more than twice. 
Corpus Christi also produced a small increase in the proportion of contacts that took place at the 
participant’s home. In Fort Worth, while there was not an increase in reported contacts at the 
workplace, there was an increase in the proportion of respondents who said that ERA staff were 
also more likely to talk with their employer “more than two times.” There were no impacts on 
these measures in Houston. 

Overall, the proportion of respondents in the ERA group who reported staff contacts at 
or with their employer is low. This is not surprising, given that many ERA group members did 
not reach the stage in the program where they were receiving the stipend and this type of 
postemployment service, as discussed in Chapter 2. In addition, this activity may not be fully 
captured by the 12-month survey because stipend receipt (and, therefore, employer visits) oc- 
curred after more than a year in the program for some individuals. 



Impacts on Service Receipt 

Table 3.2 presents information on areas in which individuals in the ERA and control 
groups received help during the year after random assignment. As shown at the top of the ta- 
ble’s second page, the ERA programs in Corpus Christi and Fort Worth produced increases in 
the proportion of ERA clients who received help with retention and advancement services, ac- 
cording to survey respondents. These impacts were 14.5 percentage points in Corpus Christi 
and 18.9 percentage points in Fort Worth. These services included help finding a better job 
while working, enrolling in life skills classes while working, and career assessment. No statisti- 
cally significant differences in retention and advancement services were found overall in Hous- 
ton, although there was a statistically significant increase in participation in activities while 
working. (Box 3.2 gives more information about how to read the tables in this report.) 

Despite the program’s impacts, the overall level of receipt of retention and advance- 
ment services is somewhat lower than expected: Only about 40 percent of working survey re- 
spondents from the ERA program reported receiving assistance in this area. This could possibly 
reflect that — because some individuals did not receive the stipend and more intensive retention 
and advancement services until they had spent more than a year in the program (see Chapter 2) 
— this activity was not fully captured by the 12-month survey. 
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Table 3.2 
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Corpus Christi Fort Worth Houston 

ERA Control Difference ERA Control Difference ERA Control Difference 

Outcome (%) Group Group (Impact) Group Group (Impact) Group Group (Impact) 
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“Employment is calculated using the ERA 12-Month Survey and includes those who reported employment since random assignment. It includes 
fonnal employment and "odd jobs." 

“This was measured only in the Texas sites. 




Box 3.2 



How to Read the Tables in This Report 

Most tables in this report use a similar format, illustrated below. The top panel shows a series of partici- 
pation outcomes for the ERA group and the control group in Houston. For example, the table shows 
that about 80 (80.2) percent of the ERA group members and about 69 (68.5) percent of the control 
group members participated in a job search activity. 

Because individuals were assigned randomly either to the ERA program or to the control group, the ef- 
fects of the program can be estimated by the difference in outcomes between the two groups. The “Dif- 
ference” column in the table shows the differences between the two research groups’ participation rates 
— that is, the program’s impacts on participation. For example, the impact on participation in a job 
search activity can be calculated by subtracting 68.5 from 80.2, yielding 1 1.6. 

Differences marked with asterisks arc “statistically significant,” meaning that it is quite unlikely that the 
differences arose by chance. The number of asterisks indicates whether the impact is statistically sig- 
nificant at the 1 percent, 5 percent, or 10 percent level (the lower the level, the less likely that the impact 
is due to chance). For example, as shown below, the ERA program had a statically significant impact of 
11.6 percentage points at the 5 percent level on participation in a job search activity. (One asterisk cor- 
responds to the 10 percent level; two asterisks, the 5 percent level; and three asterisks, the 1 percent 
level.) 

The bottom panel shows the participation outcomes among those who participated in each activity in 
the two research groups. Measures shown in italics are considered “nonexperimental” because they in- 
clude only a subset of the full report sample. Because participants in the ERA group may have different 
characteristics than participants in the control group, differences in these outcomes may not be attribut- 
able to the ERA program. Statistical significance tests are not conducted for these measures. 

Impacts on Participation in Job Search, Education, and Training Activities 



Houston 




ERA 


Control 


Difference 


Outcome (%) 


Group 


Group 


(Impact) 


Participated in a job search activity 


80.2 


68.5 


11.6 


Group job search/job club 


60.8 


50.9 


9.8 


Individual job search 


68.1 


59.1 


8.9 


Among those who participated in each type of activity: 
Average number of weeks participating in 








Job search activities 


8.8 


7.9 


0.8 


Education/training activities 


17.3 


12.1 


5.2 


Unpaid work/subsidized employment 


19.2 


13.2 


6.1 
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Overall, Table 3.2 also confirms that the major treatment difference between ERA and 
Choices appears to have occurred at the postemployment stage. There were few differences for 
most activities that took place primarily during the preemployment stage — particularly, help 
with job preparation or job search. Again in part showing the relative strength of the Choices 
program, ERA participants generally did not receive more assistance than control group mem- 
bers with such support services as child care and transportation (except in Corpus Christi) or 
more help with such basic needs as housing and public benefits. 

Table 3.3 shows the percentages of the ERA and control group members who participated 
in various employment-related activities during the year after they entered the study. Participation 
rates in employment-related services were relatively high in all sites for both the ERA group and 
the control group. For example, the proportion of the control group who participated in some type 
of services ranged from 68 percent in Corpus Christi to 78 percent in Houston. As the table indi- 
cates, only the ERA program in Fort Worth produced a statistically significant increase (of 12 per- 
centage points) in the overall participation rate in employment and education activities. 

As Table 3.3 shows, the ERA programs produced some increases in participation levels 
in certain types of employment and education services — above and beyond the control group 
— although there was variation across the sites. In Fort Worth and Houston, there were in- 
creases in the use of group job search activities. As discussed in Chapter 2, group job search 
activities were stressed somewhat more strongly in ERA than in Choices, with most sites de- 
signing a job readiness class specifically for ERA participants. In Corpus Christi and Houston, 
there were increases in the use of employment and education activities while working. This 
could reflect either the requirement that individuals participate in an employment-related activ- 
ity each month in order to receive the stipend or the option to combine work and school while 
receiving the stipend. 

Table 3.4 shows that, for the most part, the ERA group in all three sites was no more 
likely than the control group to report receiving mental health, domestic violence, or substance 
abuse services. The one exception is that there was an impact of 4 percentage points on the re- 
ceipt of domestic violence services in Houston, where a staff person from a domestic violence 
organization was colocated at the ERA program. 
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The Employment Retention and Advancement Project 
Table 3.3 
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(continued) 




Corpus Christi Fort Worth Houston 

ERA Control Difference ERA Control Difference ERA Control Difference 

Outcome Group Group (Impact) Group Group (Impact) Group Group (Impact) 
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Chapter 4 



The Effects of the Texas ERA Program on Employment, 
Public Assistance, and Income 



This chapter uses administrative records to examine whether the Texas Employment 
Retention and Advancement (ERA) program resulted in better job-finding, employment reten- 
tion, and advancement outcomes than Choices, the state’s standard welfare-to-work program for 
recipients of Temporary Assistance for Needy Families (TANF). Administrative records are 
also used to determine whether the additional services and incentives offered by ERA relative to 
Choices have had any effect on public assistance receipt or total income. As noted previously, 
control group members were not eligible for ERA services but could receive services through 
Choices and through other programs and agencies in their area. 

Using data from the ERA 12-Month Survey, this chapter examines whether ERA in- 
creased the percentage of sample members who found jobs that had better characteristics, such as 
higher wages and more fringe benefits. Findings are presented for the full report sample, for an 
early cohort, and for subgroups. As in Chapter 3, all the findings are presented separately by site: 
Corpus Christi, Fort Worth, and Houston. 1 For the full report sample, two years of follow-up are 
available for outcomes created from unemployment insurance (UI) wage data, and six quarters of 
follow-up are available on measures of TANF, food stamp receipt, and total measured income. 



Key Findings 

• The Texas ERA program did not produce consistent or large effects on em- 
ployment outcomes during the first two years of the study. During the two- 
year follow-up period, the ERA programs in Corpus Christi and Fort Worth 
led to increases above the control group’s average on some measures of em- 
ployment and employment retention. However, differences on most out- 
comes are small and not statistically significant. In Houston, the program had 
no effect on employment and earnings. 

• ERA group members in Corpus Christi received more in combined income 
from earnings, public assistance, and the monthly stipend (discussed below) 
than control group members received from earnings and public assistance 
alone. There were no effects on combined income in Fort Worth or Houston. 



'Although the Texas program operated in various cities that are called “sites” in this report, Texas counts 
as a single ERA site. 
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• In Corpus Christi and Fort Worth, the two-year impacts were concentrated 
among sample members with recent employment history prior to random as- 
signment — people who were more likely to work after random assignment 
and (among ERA group members) more likely to receive the stipend. ERA 
did not raise employment or earnings above control group levels for sample 
members with no recent employment. 



The Expected Impacts of the ERA Program 

A program like ERA can increase job-finding, employment retention, and advancement 
in a number of ways. The ERA stipend, which was available after four months of employment 
for those working a minimum of 30 hours per week, 2 would be expected to lead to improved 
retention and advancement outcomes and to higher total income. The $200 per month stipend 
was a source of income that was available only to the ERA group, and it could increase total 
income as much as $2,400 for a one-year period. According to the Texas Workforce Commis- 
sion (TWC), the goal of the stipend was to provide a work support that would enable welfare 
leavers to resolve problems that might prevent them from maintaining employment, such as 
difficulty covering child care, transportation, and other job-related expenses. Thus, if this goal 
were achieved, ERA group members should have better employment retention outcomes. 3 Also, 
by “making work pay,” the stipend should provide an incentive for TANF recipients to work. 
The stipend would motivate an individual to look harder for a job or, even, to accept a job that 
one might otherwise forgo. That said, the maximum welfare grant in Texas was rather low, 4 so 
the incentive to work in Texas (regardless of the ERA stipend) was already stronger than in 
higher-grant states. 

The 30-hour minimum work requirement might encourage some ERA group members 
to upgrade from part-time to full-time employment if the stipend were marketed early and often 
prior to employment. 5 

ERA’s preemployment services might not generate impacts because, as discussed in 
Chapter 2, the ERA group is being compared with a control group that was engaged in a well- 
established welfare-to-work program (Choices) that strengthened throughout the study period. 



2 The stipend was also available for participants who were employed 15 hours per week in combination 
with an education and training activity for an additional 15 hours per week. 

’Texas Council on Workforce and Economic Competitiveness (2000). 

4 The maximum welfare grant in Texas in 2003 was $201 for a family of three. 

5 Because of the 30-hour eligibility rule, however, there is the possibility of what economists call an “income 
effect,” whereby stipends might encourage some individuals who are already working full time to cut back from, 
for example, a 40-hour workweek to a 30-hour workweek. Also, the ability to receive the stipend while working 
only 15 hours a week and going to school the other 15 hours might encourage some people to work less. 
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Thus, the impact analysis measures the value added of ERA over a substantial and well- 
developed system of services and supports with a strong employment focus. While the partici- 
pation analysis found important differences between the ERA and control groups, these partici- 
pation differences are not large compared with similar programs analyzed in the past 6 — which 
raises the possibility of small effects on employment, retention, and advancement. However, the 
fact that ERA is being compared with a strong control group program should be helpful for 
states that already have strong welfare-to-work programs and that are deciding whether it is 
worthwhile to invest in Texas’s mix of postemployment services and a stipend. 

ERA’s postemployment services were designed to help former welfare recipients stay 
employed and increase their wages and benefits over time. Such postemployment services as 
employer site visits and monitoring and supportive services should promote improved retention 
and advancement outcomes. These improvements would likely translate into a reduction in wel- 
fare use and recidivism. Retention would also be expected to be promoted through more fre- 
quent contact with ERA staff (and with the attendant work supports, problem solving, and re- 
employment services). Increases in retention might be expected to emerge later in the first year. 
Impacts on advancement, however, often require promotions or job-changing, which takes time. 
Impacts on advancement may not be detected until Year 2 or later. 

As discussed in Chapter 1, random assignment in Texas took place at the time of TANF 
application or recertification. Thus, the study sample includes individuals whose applications 
were denied and who never received TANF, the stipend, or ERA employment services — 
which may weaken the effects of the program. Also, to a lesser extent, the inclusion of exempt 
individuals might weaken the impacts, since their participation in ERA was voluntary. 7 



Data Sources and Samples 

Unemployment insurance (UI) wage data, public assistance payment records, and ERA 
program tracking data are the primary data sources for creating outcomes of employment, earn- 
ings, TANF, food stamps, and stipend receipt and for estimating impacts on these outcomes. 



6 The differences between ERA group members and control group members in participation in job search 
and receipt of case managers’ help in obtaining work supports or advancing to a better job are modest (below 
20 percentage points) 

7 Some individuals who had a child younger than age 1, who were ill or disabled, or who were caring for a 
disabled family member were exempt from participation in the program and did not face sanctions for non- 
compliance. A separate analysis, presented in Appendix Table E.13, found that the impacts were not greatly 
diluted by including exempt sample members. Further analysis found that the employment impact estimates 
are only slightly stronger when only those whose welfare applications were accepted are included in the sam- 
ple. On a related point, the subgroup analysis presented at the end this chapter provides a rough proxy for what 
the impacts may have looked like had random assignment been conducted at the postemployment phase. 
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Quarterly earnings records are available for three years prior to random assignment and two 
years after random assignment for a total of 4,288 sample members (2,137 in the ERA group 
and 2,151 in the control group), who were randomly assigned from October 2000 through June 
2002. 8 This represents three-quarters of the eventual sample that will be analyzed in Texas. 9 For 
this same cohort, monthly public assistance records are available for two years prior to random 
assignment and only six quarters after random assignment. 10 In this report, “Year 1” refers to the 
first through fourth quarters following the quarter of random assignment. “Quarter 1” refers to 
the quarter of random assignment. Because Quarter 1 contains some earnings, TANF payments, 
and food stamp payments from the months and weeks immediately preceding random assign- 
ment, it is excluded from the summary measures of the first year of follow-up. 11 

The UI wage data are a reliable source for estimating employment and earning impacts 
because UI wage records are stored in computerized systems shortly after the completion of a 
quarter and most employers are required to submit them. UI records do, however, miss wages 
not reported to the UI system in Texas, 12 and they do not measure job characteristics. For these 
reasons, data from the ERA 12-Month Survey are also used. However, UI wage records are re- 
liable for jobs that are covered by the UI system. 

For the ERA 12-Month Survey, MDRC selected a random sample of adults in single- 
parent families who spoke either English or Spanish and who were randomly assigned from Janu- 
ary through June 2002 in Corpus Christi and Houston and from September through December 
2002 in Fort Worth. As noted in Chapter 1, the survey was administered to 775 sample members 
across the three sites, approximately 12 months following random assignment, and it achieved 
response rates of 82 percent in Corpus Christi, 75 percent in Fort Worth, and 80 percent in Hous- 
ton. (See Appendix F for further details on the survey response analysis.) The survey also has 
limitations. Individuals may recall incorrectly or may misreport some of the outcomes. Because 
the survey sample is smaller than the administrative records sample, results from the survey are 
less reliable. It is therefore more difficult to detect statistically significant impacts among the sur- 
vey samples. Furthermore, because the survey samples were drawn from only six months of the 
random assignment period in Corpus Christi and Houston — and from only three months in Fort 

"Because there is a lag in employers’ reporting to their state UI programs, earnings data obtained from 
Texas in November 2004 (and used for this analysis) cover the period through Quarter 2 of 2004. In order to 
analyze results over a two-year follow-up period, the sample had to be limited to those who were randomly 
assigned through June 2002. Welfare and food stamp data cover the period from October 1998 to January 
2004. Stipend data cover the period from October 2000 to August 2004. 

9 Seven-quarter impacts for the full research sample are shown in Appendix Table E.6. 

10 Two and a half years of stipend receipt data are also available for this cohort. 

1 'This is hue because UI wage information is available only in calendar quarters. For example, if someone 
was randomly assigned in March 2002, the quarter of random assignment is Quarter 1 of 2002, which contains 
earnings for January, February, and March. 

12 These include “off the books,’’ federal, out-of-state, and military jobs and self-employment. 
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Worth — the survey findings may not be generalizable to the entire report sample. In particular, 
the Fort Worth survey sample and report sample do not overlap at all. 



Impacts for the Full Report Sample 

Table 4.1 summarizes the impacts of ERA in each of the three sites in which the pro- 
gram was evaluated. 13 Impacts are presented for two years of follow-up on measures of em- 
ployment and earnings that were created from UI wage records. The set of three columns at the 
left of the table shows the outcomes and impacts in Corpus Christi. The first column shows the 
average value for each outcome for the ERA group, and the second column shows the average 
value for each outcome for the control group. The control group outcomes represent the bench- 
marks against which the ERA program is being compared. Because of random assigmnent, the 
control group outcomes represent what would have been expected for ERA group members if 
only Choices and not ERA had been implemented. The third column in the set shows the ef- 
fects, or impacts, of the ERA program in Corpus Christi. Impacts are calculated as the differ- 
ence in average outcomes between the ERA group and the control group. 14 The presence or ab- 
sence of “stars” (asterisks) in the impact column indicates whether or not a difference is statisti- 
cally significant. 15 Since random assigmnent ensures that there are no systematic differences 



13 Most of the impact analysis was conducted separately by site. This decision was made during the early im- 
plementation analysis, when it became clear that the implementation of ERA differed by site. Even though the 
impacts ultimately did not differ very much by site, it was decided that the analysis should be conducted based on 
prior expectations rather than on the pattern of impacts. Pooled impacts for the key tables are presented in Appen- 
dix E. Appendix Table E. 10 shows the pooled impacts on earnings and employment for Corpus Christi and Fort 
Worth (the two sites that implemented the stronger postemployment programs); Appendix Table E.ll shows 
these same impacts for all three sites combined. The pooled impacts tables show that some effects on employment 
and employment retention, although numerically small, are statistically significant in the larger pooled samples. 

14 The impacts are estimated using linear regression, which controls for a range of background characteris- 
tics, including gender, race/ethnicity, age, education, number of children, child age, prior earnings and em- 
ployment, and prior TANF and food stamp receipt. These regression-adjusted impact estimates control for the 
very small residual measured differences in sample members’ pre-random assignment characteristics that were 
not eliminated by random assignment. This helps to improve the precision of the impact estimates. For exam- 
ple, in Corpus Christi, the two-year adjusted impact on earnings was $512. The unadjusted impact on earnings 
(Appendix Table E.15) was $219. In both cases, the differences are not statistically significant. In this case, the 
differences arise, in part, because ERA group members entered the study with earnings that were approxi- 
mately $315 lower than control group members in the year prior to random assignment. The regression ad- 
justment accounts for this, and in doing so, improves the precision of the impact estimate. 

^Statistical significance is used to assess the likelihood that an ineffective program would have generated 
effects of a given size. The impact analysis for ERA utilized two-tailed T-tests to measure statistical signifi- 
cance. In the results of this report, an effect is said to be statistically significant at the 10 percent level if there is 
less than a 10 percent chance that the estimated effect could have stemmed from a program that had no real 
effect. Statistical significance is also presented at the 5 percent and the 1 percent levels. Unless otherwise noted, 
all impacts — or “increases” or “decreases” — are statistically significant. 
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The Employment Retention and Advancement Project 
Table 4.1 
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Corpus Christi Fort Worth Houston 

ERA Control Difference ERA Control Difference ERA Control Difference 

Outcome Group Group (Impact) Group Group (Impact) Group Group (Impact) 
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between the ERA and control groups, other than exposure to the program being studied, any 
statistically significant differences in outcomes after random assignment can be attributed to the 
ERA program. The sets of columns in the middle and at the right of Table 4. 1 follow the same 
fonnat but show the impacts of ERA in Fort Worth and Houston, respectively. 

Benchmark Employment, Retention, and Earnings Outcomes 

According to data from the baseline information form, nearly 94 percent of study par- 
ticipants were unemployed at the time of random assignment. Despite this high percentage, 
most control group members in Texas worked at some point after random assignment. As 
shown in Table 4.1, in Corpus Christi, nearly 85 percent of control group members worked in a 
Ul-covered job sometime in the first two years after random assignment. Control group mem- 
bers were somewhat less likely to work in Fort Worth (76 percent) and Houston (72 percent). 

One of the main goals of ERA was to promote employment retention. A review of em- 
ployment retention outcomes for the control group provides compelling evidence of the need for 
an intervention like ERA. While some employment was common, few control group members 
were stably employed. Across the sites, between 42 percent and 50 percent of the control group 
worked in Ul-covered jobs in any given quarter. 16 Among control group members, 20 percent 
who were employed in Quarter 2 did not work in Quarter 3 (not shown). Only about 35 percent 
of control group members who were working in Quarter 2 worked in each of the quarters 
through the last quarter of Year 2. Though many Texas sample members eventually found jobs, 
overall employment rates in Texas were no higher in Year 2 than Year 1 . 

Earnings among control group members were low for all three sites and throughout the 
follow-up period. This is partly attributable to low employment rates. (Table 4. 1 and all tables in 
the report include zeroes for those who were not working, unless otherwise specified.) How- 
ever, in any given quarter, employed control group members in Corpus Christi earned an aver- 
age of only $2,000 (not shown). 

Two-Year Impacts on Employment, Employment Retention, and Earnings 

The first section of Table 4.1 shows the impacts over the first two years after random 
assignment. This table, like most of the tables in this report, shows little evidence that ERA 
made a difference in the employment and earnings outcomes of sample members. 

The ERA program in Corpus Christi did not produce any statistically significant effects 
on employment, employment retention, or earnings over the two-year follow-up period. ERA 

16 The average quarterly employment measure was computed by adding up the number of quarters em- 
ployed and dividing by the total number of quarters potentially employed. 
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had no effect on the percentage ever employed. The program did not increase the percentage of 
ERA group members who were employed above the control group’s benchmark of 84.7 per- 
cent. The lack of an effect on employment is likely attributable to the fact that, as implemented, 
the ERA program was similar to the Choices program during the preemployment phase. This 
shows that the stipend did not induce individuals to work who would not have worked anyway. 
During Years 1 and 2, the ERA group in Corpus Christi did not earn significantly more than the 
control group’s average of $8,088. 

Table 4.1 shows that, during Year 1, ERA produced increases in measures of employ- 
ment retention in Corpus Christi. ERA increased average quarterly employment by nearly 4 
percentage points above the control group’s average of approximately 50 percent. ERA in- 
creased the proportion of program group members who were employed in four consecutive 
quarters — a key measure of retention — by about 4 percentage points above the control 
group’s average of 26 percent. Despite the increases in employment stability, the program did 
not increase earnings. Given the variability in earnings, it is occasionally the case that effects on 
employment rates are statistically significant while effects on earnings are not. 17 

In Year 2, the impacts on measures of employment retention were no longer statistically 
significant. This contradicts the expectation that the impacts would be stronger later in the fol- 
low-up period and that they would vary based on stipend receipt. In Corpus Christi, only about 
16 percent of the ERA group received a stipend in Year 1 (not shown). In Year 2, approxi- 
mately 24 percent of the ERA group received a stipend, but the impacts on employment reten- 
tion measures were no longer significant. 

The impacts on quarterly employment and earnings are shown in Appendix Table E.5. 
In Corpus Christi, there are several quarters in which employment differences, although posi- 
tive, are not statistically significant. Table 4.1 shows, however, that impacts on quarterly em- 
ployment emerged late in Year 2 in Corpus Christi. In Quarter 9, ERA increased the percentage 
employed in Corpus Christi by 5.1 percentage points; however, much of this effect on employ- 
ment was in jobs with very low earnings (less than $500 per quarter). These low earnings may 
be attributable to part-time jobs, low wages, or short-term employment. 18 As a result, ERA 
group members did not earn more, on average, than control group members. It is difficult to 



l7 The impact on total earnings in Year 1 was close to being statistically significant (p-value = 0.154). 
There is one year of follow-up data available for the full sample (including individuals randomly assigned after 
June 2002). The first-year impact on total earnings is slighdy larger among this sample (approximately $400) 
and is statistically significant at the 10 percent level, partly due to the somewhat-larger sample size. 

ls Most likely, sample members who had earnings this low did not work consistently through the quarter, 
but it is impossible to know for sure, since UI wage data are collected only quarterly. Impacts on employment 
at different levels of earnings are shown in Appendix Table E.5. 
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determine whether this late effect is a temporary phenomenon, but the fact that none of the im- 
pacts in the previous quarters are close to significant suggests that this impact may not persist. 19 

In Fort Worth, nearly 81 percent of ERA group members were ever employed over the 
two-year follow-up period, which is 4 percentage points higher than the control group’s aver- 
age. This impact is consistent with the participation results, discussed in Chapter 3, which show 
that ERA increased participation in group job search activities in Fort Worth (which was not the 
case in Corpus Christi). This impact may also suggest that, unlike in Corpus Christi, the stipend 
did have a modest effect on employment in Fort Worth. There were no other significant impacts 
on employment or earnings over the two-year follow-up period in Fort Worth. Over that period, 
average earnings for the two research groups were approximately $9,500. 

In Year 1, there were no impacts in Fort Worth on any of the measures of employment 
and earnings shown in Table 4.1. The weak results in Year 1 may be due to the startup problems 
in Fort Worth (noted in Chapter 2). In contrast to the pattern of impacts in Corpus Christi, the 
impacts in Fort Worth were stronger in Year 2, when nearly 69 percent of ERA group members 
were employed — 6 percentage points more than control group members. This impact emerged 
because the employment rate among ERA group members was the same in Year 2 as Year 1 
while the percentage of control group members ever employed in Year 2 declined. However, 
there were no effects on measures of job retention or earnings. 

By the end of Year 2, the impacts in Fort Worth were stronger than they were earlier in 
the follow-up period. Though ERA no longer had a statistically significant impact on the em- 
ployment rate, the program produced a statistically significant $240 increase in earnings in the 
last quarter of Year 2. ERA increased the percentage of sample members who had earnings of 
$2,500 or more — an important measure of advancement. While these results are promising, it 
is impossible to know whether they represent a statistical anomaly or a real trend. Future reports 
will provide more definitive results. 

In Houston, ERA had no effect on measures of employment, retention, or advancement. 
Over the two-year follow-up period, ERA group members in Houston earned $8,269, which is 
nearly the same as control group members earned. The two groups’ rates of employment and em- 
ployment retention were nearly identical over the follow-up period. In any given quarter, approxi- 
mately 42 percent of ERA and control group members were employed, which is somewhat lower 
than in Corpus Christi and Fort Worth. The consistent lack of impacts for several measures, sub- 
groups, and cohorts suggests that it is unlikely that full-sample impacts will emerge in Houston. 

19 Impact estimates are available for an early cohort through Year 3. In this cohort, the impacts also became 
stronger in the last quarter of Year 2 and were sustained throughout Year 3. However, the impacts for this co- 
hort tended to be somewhat larger than for the full sample, and the sample size is smaller — so it is not yet 
clear what the Year 3 impacts will look like for the full sample. 
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Impacts on Stipend Receipt, Public Assistance, and Income 

As discussed, TANF and food stamp data are available only through January 2004, and 
so two fewer quarters of follow-up are available from these sources. For this reason, the meas- 
ures in the upper panel of Table 4.2 cover six quarters after random assigmnent (one and a half 
years) rather than two years. 20 Table 4.2 shows that, over the follow-up period, ERA group 
members in Corpus Christi received more in combined income from earnings, public assistance, 
and the stipend than control group members received from earnings and public assistance alone. 
Receipt of the stipend accounts for most of the difference. There were no effects on combined 
income in Fort Worth or Houston. 

Approximately 82 percent of control group members in Texas ever received TANF in 
the first six quarters. This is important, since ERA’s pre- and postemployment services — and 
the stipend — were available only to those who received TANF. Although the percentage ever 
receiving welfare was high, the rate declined rapidly during the follow-up period. 

Across the three sites, between 92 percent and 97 percent of control group members re- 
ceived food stamps at some time in the follow-up period. While TANF receipt rates declined 
rapidly, the food stamp receipt rates were more stable: Approximately 70 percent of sample 
members were still receiving food stamps in Quarter 7. Thus, for this sample, food stamps were 
a much more important source of income than TANF payments. 

Total measured income includes income from earnings, TANF payments, food stamp pay- 
ments, and stipends (for ERA group members only). Despite die unstable employment and low 
earnings of control group members in Texas, earnings were the primary source of income. On aver- 
age, approximately 53 percent of total income was derived from earnings; 34 percent came from 
food stamps; and TANF provided only 13 percent. Total income among control group members 
over die six-quarter period varied from $11,247 in Corpus Christi to $12,227 in Fort Worth. 21 



20 Everyone who was randomly assigned through December 2001 has TANF and food stamp data avail- 
able through Quarter 9. Sample members randomly assigned in the first quarter of 2002 are missing one quarter 
of payments, and those randomly assigned in Quarter 2 of 2002 are missing two quarters (data are available 
through Quarter 7). For this reason, the measures in this section go through Quarter 7 only (which is six quar- 
ters after the quarter of random assignment). 

21 While these estimates are far below the poverty line (given that this period is a year and a half), it is im- 
portant to note that this measure provides a substantial underestimate of total household income. A fuller ver- 
sion of income that includes income from jobs not covered by the UI system and from other household mem- 
bers, child support. Supplemental Security Income (SSI), and other sources is available from the ERA 12- 
Month Survey. This measure suggests that household income among control group members was approxi- 
mately $12,000 annually (still below the federal poverty level for a family of three, but much higher than the 
partial estimate available from the administrative records). In addition, neither measure includes an estimate of 
the Earned Income Tax Credit (EITC), an important source of income for low-wage workers. 
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The Employment Retention and Advancement Project 
Table 4.2 

Impacts on Public Assistance and Measured Income 
Texas 
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SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the State of Texas. 
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ERA did not affect the number of people who received any TANF or food stamps in 
Corpus Christi. However, because only ERA group members were eligible for the stipend, total 
income from all four sources was $604 higher among ERA group members (a 5 percent in- 
crease over the control group level). Over the two-year follow-up period, the average amount 
received from the stipend was approximately $300 (not shown). During this time period, 27 per- 
cent of the ERA group received a stipend. 22 This implies that those who received the stipend 
earned more than $1,100, on average, over the follow-up period ($300/0.27 = $1,11 1). 

By Quarter 2 of Year 2 (see the lower panel of Table 4.2), ERA produced a statistically 
significant reduction in welfare receipt in Corpus Christi. Reducing recidivism was a key goal 
of the program. By that quarter, ERA group members were nearly 5 percentage points less 
likely to receive welfare. Since it was the first quarter in which an impact was evident, it is too 
early to say whether the ERA program in Corpus Christi will continue to reduce welfare re- 
ceipt. 23 ERA no longer produced a significant impact on total income in Corpus Christi by the 
second quarter of Year 2. 

In Fort Worth, ERA had no effect on TANF, food stamp payments, or total income 
over the first six quarters. By Quarter 2 of Year 2, the ERA program in Fort Worth was still not 
producing an effect on these outcomes. The lack of an impact on income in Fort Worth is partly 
due to the lower take-up of the ERA stipend in that site. Approximately 19 percent of ERA 
group members in Fort Worth had received a stipend by Quarter 7. 

In Houston, ERA generated a small increase in the amount of TANF received over the 
six-quarter follow-up period. This impact was the result of a series of small but statistically in- 
significant increases in TANF over several quarters. However, because of the lack of an impact 
on earnings and the relatively low amount received from the stipend in Houston, there was no 
impact on total measured income. 24 

Figure 4. 1 shows the percentage who were employed and receiving the stipend and the 
percentage who were employed without receiving the stipend in each quarter of the follow-up 
period in Corpus Christi. 25 The control group’s bars show the employment rates that would have 
been expected in the absence of ERA. To the extent that the shaded portion of the ERA group’s 
bar overlaps the control group’s bar, the stipend was paid to those who would have worked 



22 The stipend receipt estimates presented in Chapter 2 are somewhat higher because they cover a longer 
follow-up period. 

~ 3 Quarterly impacts on TANF and food stamp measures are shown in Appendix Tables E.7 and E.8. 

24 The impacts in this section were computed among single-parent families only. The only site with a suffi- 
cient sample of two-parent families to allow for a reasonable analysis is Corpus Christi. There are no statisti- 
cally significant impacts on employment or earnings in this small sample (N = 178). 

“'The comparable figures for Fort Worth and Flouston are shown in Appendix E. 
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SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the State of Texas. 



anyway. (In the study of incentive programs, this is known as “windfall.”) To the extent that the 
shaded region exceeds the control group’s employment bar, the stipends encouraged new em- 
ployment. Although the ERA group’s bars are mostly a little higher than the control group’s, the 
figure shows that stipends do not appear to have added much additional employment. While 
Figure 4.1 shows that the stipend did not achieve its employment goals, the stipend did generate 
an increase in income in Corpus Christi (Table 4.2). 

Impacts on Job Characteristics 

Table 4.3 summarizes the impacts of the Texas ERA program on several measures of 
job characteristics, based on the ERA 12-Month Survey. 26 It shows that, for the most part, ERA 
had no effect on the characteristics of participants’ jobs. 

In addition to the cautions raised about the representativeness of the survey cohorts (see 
“Data Sources and Samples,” above), two new cautions emerged from reviewing the findings that 
are based on administrative records. First, the ERA 12-Month Survey covers only Year 1, which 
is a period when only 16 percent of the ERA group had received a stipend in Corpus Christi (for 
example). Second, Table 4.1 shows that both ERA and control group members who worked had 
higher average earnings in Year 2 than in Year 1 . For these reasons — and the fact that the survey 
is drawn from fairly narrow cohorts — the Year 1 results may not be representative. 

Most control group members were working in low- wage jobs. As suggested in the ad- 
ministrative records analysis, wages were especially low in Corpus Christi, where 60 percent of 
employed control group respondents worked at jobs that paid less than $7 per hour. In all three 
sites, relatively few worked in jobs with employer-provided benefits, such as sick days, dental 
benefits, and health insurance. 

In Corpus Christi, few effects are large enough to be statistically significant. Among the few 
impacts that do reach significance, perhaps the most encouraging are the effects on working in jobs 
that require important skills. ERA group respondents in Corpus Christi were more likely than those 
in the control group to work in jobs that required computer skills and that required arithmetic. 27 

In Fort Worth, the survey sample size is below 100 per research group, so the findings 
are less reliable. Although an impact is found in the administrative records, no effect on job 
placement is evident in the survey. Not much can be made of this, however, because the cohorts 
do not overlap. In Fort Worth as in Corpus Christi, ERA increased the percentage of sample 
members who worked with computers. 



26 Appendix Tables E.2 to E.4 show the impacts of ERA on other survey outcomes. 
27 Web site: http://www.fordschool.umich.edu/research/poverty/wes/index.html. 



78 




The Employment Retention and Advancement Project 
Table 4.3 

Impacts on Characteristics of Current Job 
Texas 
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(continued) 




Corpus Christi Fort Worth Houston 

ERA Control Difference ERA Control Difference ERA Control Difference 

Outcome Group Group (Impact) Group Group (Impact) Group Group (Impact) 
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(continued) 




Corpus Christi Fort Worth Houston 

ERA Control Difference ERA Control Difference ERA Control Difference 

Outcome Group Group (Impact) Group Group (Impact) Group Group (Impact) 
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In Houston, the survey impacts are more positive than would be expected based on the 
findings from the larger sample for which administrative records are available. For example, re- 
spondents in the ERA group were nearly 9 percentage points more likely to work in full-time jobs, 
which is close to being statistically significant (p- value = 0.108). This is likely due to response 
bias. Specifically, the impacts of ERA on employment and earnings from the administrative re- 
cords are more positive among Houston’s survey respondents than among its full report sample. 28 

Impacts on Employment Stability 

Table 4.4 shows outcomes related to job retention, based on the ERA 12-Month Sur- 
vey. ERA had no significant impacts on measures of job retention in any of the Texas sites. 

The control group outcomes show that between 22 percent (in Corpus Christi) and 13 
percent (in Houston) of control group respondents worked more than 10 months in the first 
year. Approximately half of those who were employed worked for 7 or fewer months. Only 
about 35 percent of control group members worked for the same employer for 6 months or 
more. Sample members were likely to work in multiple jobs in the one-year follow-up period. 
For example, in Corpus Christi, 29 percent of all control group respondents worked in two or 
more jobs; this constitutes nearly 46 percent of those who ever worked. 

In Corpus Christi, ERA produced no statistically significant effects on job retention. 
While the findings from the administrative records indicate that there were some effects on job 
retention in Corpus Christi in Year 1, these effects are not observed in the survey sample. This 
may be because the survey sample size in Corpus Christi is rather small, making it difficult to 
detect statistically significant results. In Fort Worth and Houston, there were no effects on job 
retention (similar to the findings from the analysis of administrative records). 

Impacts in Year 3 

As discussed, a smaller group of sample members who were randomly assigned through 
June 2001 — the “early cohort” — have three years of follow-up data on measures of earnings 
and employment. Results for the early cohort may provide a preview of the impacts for the full 
sample in Year 3. The sample sizes that have three years of follow-up are 668 in Corpus Christi, 
710 in Fort Worth, and 673 in Houston. Impacts for this early cohort and for the report sample are 



28 For example, for the full report sample, ERA reduced earnings in Year 1 by $94, compared with in- 
creases of $228 for the fielded survey sample and $728 for the survey respondent sample. For the full sample, 
ERA reduced the percentage working in any given quarter by 1.0 percentage point, compared with increases of 
3.7 percentage points for the fielded sample and 5.0 percentage points for the survey respondent sample. This 
response bias cannot be rectified by weighting based on background characteristics because differences in 
measurable characteristics are not systematic. 
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Table 4.4 

Impacts on Employment Retention 
Texas 
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SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 




shown in Appendix Figure E.9. In Corpus Christi, this small group of early enrollees in the pro- 
gram experienced statistically significant employment impacts in Year 3. In the other two sites, 
impacts did not vary across cohorts. It should be noted that although the impacts look positive in 
Year 3 for this early cohort in Corpus Christi, there is no assurance that the impacts for the full 
report sample will be positive. Indeed, for the overlapping quarters, the impacts among the early 
cohort are somewhat larger and more consistent than the impacts for the full report sample. 



Impacts for Subgroups Based on Employment in the Prior Year 

An analysis of stipend receipt rates, discussed in Chapter 2, found that certain sub- 
groups of the ERA group were more likely to receive the ERA stipend. In particular, groups that 
had more recent employment history were more likely to receive the stipend, because they were 
more likely to work in the post-random assignment period. Thus, it might be expected that the 
impacts of ERA in Texas would be stronger among subgroups that had recent employment his- 
tory. It is also possible that an intervention like ERA is more effective for those who are more 
easily employable. 

In Corpus Christi and Fort Worth, the two-year impacts were concentrated among sample 
members who had recent employment history prior to entering the study. For those without recent 
employment, there were no impacts on employment or employment retention. In Corpus Christi, 
ERA increased the percentage of sample members who were employed four consecutive quarters 
by nearly 6 percentage points among those who had worked in the year prior to entering the study. 
ERA did not raise employment or earnings above control group levels for sample members with 
no recent employment. 29 The impact on employment in Fort Worth was also concentrated among 
those with recent employment. In Houston, there were no impacts in either subgroup. 

This pattern of subgroup impacts may be attributable to two factors. First, the sample 
members who were employed in the year prior to entering the study were reemployed sooner 
and, thus, more quickly reached the postemployment phase (where the difference between ERA 
and Choices was largest). Second, these sample members may have been more likely to have 
the necessary skills and human capital to better utilize the postemployment services and sup- 
ports that the ERA program provided. These results are presented in Appendix Table E. 16. 



29 Further analysis found that the difference in employment impacts across the subgroups in Corpus C hristi 
are statistically significant. 
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Supplementary Table for Chapter 1 
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Appendix Table A.l (continued) 
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Appendix Table A.l (continued) 
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Appendix B 

Notes for Tables and Figures Displaying Results 
Calculated with Administrative Records Data 




Estimates were regression-adjusted using ordinary least squares, controlling for pre-random as- 
signment characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 
1 percent. 

Italics indicate comparisons that are nonexperimental. These measures are computed only for 
sample members who were employed. Since there may be differences in the characteristics of 
program group and control group members who were employed, any differences in outcomes 
may not necessarily be attributable to the ERA program. Statistical tests were not perfonned. 

“Year 1” refers to Quarters 2 to 5. Quarter 1 is the quarter in which random assignment took 
place. 

Dollar averages include zero values for sample members who were not employed or were not 
receiving TANF or food stamps. 

Unless otherwise stated, results are for sample members randomly assigned from October 2000 
to June 2002. 

NA = not applicable. 
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Appendix C 

Examples of Employment-Related Measures 
Analyzed in This Report 




For this report on the Employment Retention and Advancement (ERA) program in 
Texas, employment-related measures are created from unemployment insurance (UI) wage re- 
cords and the ERA 12-Month Survey. This appendix describes some of the key measures in 
more detail, grouping them by the research questions that they help to answer. Measures from 
both UI wage records and the 12-month survey are discussed. 1 



Did ERA Improve Job Placement? 

According to the baseline survey, nearly all the sample members in Texas were without 
work at the time of random assignment. Thus, an important measure of the effectiveness of the 
program is how well ERA placed sample members in jobs. 

• Ever employed in a Ul-covered job in Years 1-2. This is a key measure of 
job placement. A preemployment program like ERA in Texas is expected to 
produce increases in this measure. It captures both the movement from no 
employment to Ul-covered employment and the movement from jobs not 
covered by the UI system to jobs that are covered. In general, jobs that are 
covered by the UI system are more likely to have benefits and to count to- 
ward eligibility for the Earned Income Tax Credit (EITC). 

• Ever employed as of the 12-month survey interview. Based on responses 
to the ERA 12-Month Survey, this is a more comprehensive measure of job 
placement that captures the movement from no employment to either Ul- 
covered jobs or jobs not covered by the Ul-system. The shortcomings of this 
measure are that it is based on recall of past events, rather than on administra- 
tive records, and that it is based on the smaller survey sample. 2 



Did ERA Improve Employment Retention? 

As discussed in the introduction to the report, although a lot is known about how to place 
welfare recipients in jobs, very little is known about how to help them retain those jobs. Facing a 
variety of barriers to work — including health issues, unreliable or costly child care and transpor- 



ts discussed in the report, UI wage data are a reliable source for estimating employment and earnings 
impacts because UI wage records are stored in computerized systems shortly after the completion of a quarter 
and most employers are required to submit them. UI records do, however, miss wages that are not reported to 
the UI system in Texas, including “off-the-books,” federal, out-of-state, and military jobs and self-employment. 
Also, UI records usually do not measure job characteristics. For these reasons, data from the ERA 12-Month 
Survey are also used. 

2 The advantages and disadvantages of survey measures versus UI measures are the same for all the items 
listed in this appendix and thus are not repeated throughout. 
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tation arrangements, and difficult working conditions — welfare recipients often have unstable 
employment experiences. One of the key goals of ERA in Texas was to stabilize employment. 

• Average quarterly employment in Ul-covered jobs. This measure can be 
defined as the employment rate in the average quarter, and though it is re- 
lated to employment retention, it might also reflect job placement and the 
timing of initial employment. Average quarterly employment was computed 
by adding up the number of quarters employed and dividing by the total 
number of quarters potentially employed. For example, for the two-year 
measure, a sample member who was employed in two quarters received a 
value of 25 percent [(2/8) * 100], 

• Employed four consecutive quarters in Ul-covered jobs. An impact on 
this measure would likely signal an effect on employment retention. Because 
UI wage records are reported quarterly, it is not possible to know whether 
sample members who worked in four consecutive quarters were really em- 
ployed the whole time. 

• Number of months employed since random assignment. This item, from 
the 12-month survey, is a measure of employment stability. While it is simi- 
lar to the two items above, it provides a more finely grained measure of em- 
ployment stability, since survey data can be collected at monthly intervals. 
Similarly, the survey item “employed six consecutive months” is a measure 
of employment stability that is comparable to “employed four consecutive 
quarters,” but it provides a better estimate of stability because it is based on 
months rather than quarters. 



Did ERA Lead to Advancement in the Labor Market? 

The goals of ERA go beyond employment retention. Retention at a low-wage or low- 
quality job may represent some improvement, but the goals of ERA included advancement to 
jobs that offered better pay and benefits. Improvements in job quality can be viewed as a type of 
advancement. Some of these measures are mostly noneconomic (such as whether one works the 
night shift) but are still important. Others (such as health benefits) can have large economic con- 
sequences that are not incorporated in measures of earnings. 

• Earned over $10,000. This measure could be related to both retention and 
advancement, although — like some of the other measures — it could also 
reflect the timing of initial employment. The two-year measure of earnings 
uses a threshold of $20,000. 
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• Earnings distribution in a quarter (earned $2,500 or more; earned be- 
tween $500 and $2,499; earned between $1 and 499). This measure shows 
whether increases in Ul-covered employment are driven by increases in em- 
ployment at certain levels of earnings. It is likely related to advancement and 
job quality, although it is subject to the limitations of UI earnings data that 
are described above. 

• Employed at a good job. A “good job” is a job in which a respondent works 35 
or more hours per week and either (1) pays $7.00 or more per hour and offers 
health i ns urance or (2) pays $8.50 or more per hour. 3 By coupling wages and 
benefits, this measure allows for a more nuanced assessment of job quality. 

• Job schedule measures. For workers in general, and for working mothers 
with young children in particular, the job schedule can raise critical issues. It 
can be difficult, for example, to arrange for child care during the evening, 
and overnight shifts can be even more difficult to accommodate. On the other 
hand, such atypical schedules may command higher wage rates. For these 
reasons, it is important to analyze job schedules. 

• Job skills index. These survey measures were adopted from the Woman’s 
Employment Study (WES). 4 Working at jobs that require skills for which 
there is a high demand in the labor market is an important pathway to ad- 
vancement. Even if these skills are not compensated for immediately, they 
may lead to longer-term improvement in labor market outcomes. 

• Employer-provided benefits. The availability of benefits is obviously im- 
portant. A lower-wage job with such key benefits as health and dental insur- 
ance may be more economically beneficial than a higher-wage job without 
benefits. On the other hand, many sample members were eligible for Medi- 
caid, which may have provided more affordable benefits than private em- 
ployer-based health insurance programs. An important point is that the 
measures presented in this report reflect self-reported assessments of the 
availability of benefits. Sample members may have elected not to participate 
in benefit plans, particularly if the plans were too expensive. 



3 This definition of a good job is adapted from Johnson and Corcoran (2003). 

4 Web site: http://www.fordschool.umich.edu/research/poverty/wes/index.html. 
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What Was the Overall Effect of ERA on Employment Retention 
and Advancement? 

As discussed in the introduction, ERA in Texas had many goals. The employment goals 
included initial job placement, employment retention, and advancement. 

• Earnings in Ill-covered jobs. An impact on average total earnings could re- 
flect improvements in job placement, retention, or advancement or some 
combination of the three. For this reason, impacts on total earnings are a 
comprehensive indicator of the effectiveness of ERA. 

• Hourly and weekly wages. Measures of earnings are also created from the 
ERA 12-Month Survey. The survey measures are more refined than the UI data, 
because earnings can be expressed as hourly wages or as weekly earnings. Thus, 
the survey measures provide an indication of whether any differences in earnings 
are “driven” by the number of hours worked or by the wage rates. 
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Appendix D 

Notes for Tables and Figures Displaying Impacts 
Calculated with Responses to the ERA 12-Month Survey 




Estimates were regression-adjusted using ordinary least squares, controlling for pre-random as- 
signment characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between outcomes for the program and control 
groups. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 
1 percent. 

Italics indicate comparisons that are nonexperimental. These measures are computed only for 
sample members who were employed. Since there may be differences in the characteristics of 
program group and control group members who were employed, any differences in outcomes 
may not necessarily be attributable to the ERA program. Statistical tests were not perfonned. 

NA = not applicable. 
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Appendix E 

Supplementary Tables and Figures 




The Employment Retention and Advancement Project 
Appendix Table E.l 

Impacts on Quarterly Ul-Covered Employment and Earnings for the 
Report Sample and Early Cohort, by City 

Texas 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Corpus Christi 
Report sample 2 

Ever employed (%) 


Quarter of random assignment 


49.5 


50.6 


-1.0 


Q2 


52.3 


48.9 


3.4 


Q3 


53.2 


50.4 


2.8 


Q4 


53.6 


50.1 


3.5 


Q5 


53.8 


49.6 


4.2 


Q6 


53.4 


51.1 


2.3 


Q7 


53.2 


52.9 


0.3 


Q8 


52.6 


51.1 


1.5 


Q9 


54.4 


49.1 


5.3 ** 


Earnings ($) 


Quarter of random assignment 


536 


523 


13 


Q2 


791 


755 


36 


Q3 


998 


894 


105 


Q4 


1,061 


920 


141 * 


Q5 


1,078 


1,018 


60 


Q6 


1,113 


1,057 


56 


Q7 


1,153 


1,117 


36 


Q8 


1,198 


1,167 


31 


Q9 


1,215 


1,144 


71 


Sample size (total = 1 ,309) 


656 


653 




Earlv cohort 1 * 

Ever employed (%) 


Quarter of random assignment 


52.8 


50.8 


2.1 


Q2 


55.4 


48.7 


6.8 * 


Q3 


56.2 


50.3 


5.9 


Q4 


54.1 


52.4 


1.7 


Q5 


56.2 


48.2 


8.1 ** 


Q6 


52.5 


49.9 


2.5 


Q7 


54.6 


51.1 


3.5 


Q8 


52.2 


51.4 


0.9 


Q9 


54.9 


48.3 


6.6 * 


Q10 


53.4 


45.1 


8.3 ** 


Qll 


51.6 


44.4 


7.2 * 


Q12 


50.4 


42.0 


8.4 ** 


Q13 


52.2 


42.4 


g g *** 



(continued) 
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Appendix Table E.l (continued) 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Earnings ($) 


Quarter of random assignment 


585 


550 


36 


Q2 


788 


695 


92 


Q3 


1,016 


821 


195 * 


Q4 


1,074 


843 


231 ** 


Q5 


1,049 


973 


76 


Q6 


1,075 


977 


98 


Q7 


1,091 


1,051 


40 


Q8 


1,167 


1,112 


55 


Q9 


1,156 


999 


157 


Q10 


1,116 


1,019 


98 


Qll 


1,214 


1,030 


184 


Q12 


1,257 


1,025 


232 * 


Q13 


1,152 


1,014 


138 


Sample size (total = 668) 


337 


331 




Fort Worth 
Report sample 11 

Ever employed (%) 


Quarter of random assignment 


45.6 


44.2 


1.3 


Q2 


47.5 


46.1 


1.4 


Q3 


51.4 


48.4 


3.0 


Q4 


49.1 


46.9 


2.2 


Q5 


47.5 


47.8 


-0.3 


Q6 


49.4 


45.9 


3.6 


Q7 


47.9 


47.1 


0.8 


Q8 


46.5 


45.9 


0.6 


Q9 


48.5 


44.4 


4.1 


Earnings ($) 


Quarter of random assignment 


596 


547 


48 


Q2 


847 


857 


-9 


Q3 


1,132 


1,055 


77 


Q4 


1,250 


1,136 


114 


Q5 


1,224 


1,233 


-9 


Q6 


1,228 


1,190 


38 


Q7 


1,295 


1,310 


-15 


Q8 


1,368 


1,213 


155 


Q9 


1,448 


1,212 


237 ** 


Sample size (total = 1,163) 


577 


586 




Earlv cohort” 

Ever employed (%) 


Quarter of random assignment 


47.7 


45.3 


2.4 


Q2 


49.5 


46.4 


3.1 


Q3 


54.8 


48.4 


6.4 * 



(continued) 
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Appendix Table E.l (continued) 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Q4 


49.7 


48.3 


1.4 


Q5 


48.9 


51.6 


-2.6 


Q6 


48.5 


48.2 


0.3 


Q7 


47.4 


48.1 


-0.7 


Q8 


43.1 


46.4 


-3.3 


Q9 


44.3 


44.4 


-0.1 


Q10 


47.2 


48.6 


-1.4 


Qll 


47.1 


45.6 


1.5 


Q12 


46.8 


41.8 


5.0 


Q13 


47.5 


43.9 


3.6 


Earnings ($) 


Quarter of random assignment 


604 


547 


58 


Q2 


911 


857 


55 


Q3 


1,231 


1,064 


167 


Q4 


1,279 


1,185 


94 


Q5 


1,266 


1,356 


-90 


Q6 


1,186 


1,238 


-52 


Q7 


1,288 


1,361 


-73 


Q8 


1,348 


1,277 


71 


Q9 


1,437 


1,338 


98 


Q10 


1,360 


1,362 


-1 


Qll 


1,487 


1,430 


57 


Q12 


1,564 


1,341 


223 


Q13 


1,638 


1,443 


196 


Sample size (total = 710) 


347 


363 




Houston 
Report sample c 

Ever employed (%) 


Qi 


39.0 


38.8 


0.1 


Q2 


41.7 


42.1 


-0.4 


Q3 


42.9 


45.8 


-2.9 


Q4 


43.0 


43.5 


-0.5 


Q5 


42.7 


42.2 


0.5 


Q6 


44.5 


40.6 


3.9 * 


Q7 


44.2 


40.8 


3.4 


Q8 


42.3 


40.8 


1.5 


Q9 


40.0 


41.6 


-1.6 


Earnings ($) 


Quarter of random assignment 


494 


537 


-43 


Q2 


732 


755 


-23 


Q3 


948 


1,022 


-74 


Q4 


1,028 


1,017 


11 


Q5 


1,084 


1,073 


11 



(continued) 
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Appendix Table E.l (continued) 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Q6 


1,144 


1,066 


78 


Q7 


1,134 


1,104 


30 


Q8 


1,113 


1,095 


18 


Q9 


1,087 


1,177 


-90 


Sample size (total = 1,816) 


904 


912 




Earlv cohort' 

Ever employed (%) 


Quarter of random assignment 


38.6 


42.0 


-3.3 


Q2 


40.8 


44.8 


-4.0 


Q3 


43.5 


47.4 


-3.9 


Q4 


40.4 


41.0 


-0.6 


Q5 


41.1 


40.9 


0.3 


Q6 


41.8 


36.9 


4.8 


Q7 


43.2 


42.1 


1.1 


Q8 


39.9 


38.8 


1.1 


Q9 


41.3 


40.7 


0.6 


Q10 


40.2 


41.6 


-1.4 


Qll 


39.7 


40.5 


-0.8 


Q12 


40.4 


40.2 


0.2 


Q13 


41.8 


36.9 


5.0 


Earnings ($) 


Quarter of random assignment 


547 


574 


-28 


Q2 


691 


771 


-80 


Q3 


935 


1,065 


-130 


Q4 


997 


941 


56 


Q5 


972 


960 


12 


Q6 


1,025 


903 


121 


Q7 


1,103 


1,061 


42 


Q8 


1,112 


1,036 


76 


Q9 


1,070 


1,048 


22 


Q10 


1,158 


1,061 


97 


Qll 


1,071 


1,088 


-16 


Q12 


1,084 


1,157 


-73 


Q13 


1,220 


1,056 


164 


Sample size (total = 673) 


340 


333 





(continued) 
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Appendix Table E.l (continued) 

SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the State of Texas. 
NOTES: See Appendix B. 

This table includes only employment and earnings in jobs covered by the Texas unemployment insurance (UI) 
program. It does not include employment outside Texas or in jobs not covered by UI (for example, "off-the- 
books" jobs, some agricultural jobs, and federal government jobs). 

a Corpus Christi and Fort Worth report sample members were randomly assigned from October 2000 to June 

2002 . 

“Corpus Christi and Fort Worth early cohort members were randomly assigned from October 2000 to June 

2001 . 

“Houston report sample members were randomly assigned from March 2001 to June 2002, and early cohort 
members for this site were randomly assigned from March to June 2001 . 
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Sample size (total = 775) 141 149 92 96 150 




Appendix Table E.2 (continued) 
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The Employment Retention and Advancement Project 
Appendix Table E.3 
Impacts on Other Outcomes 
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(continued) 




Appendix Table E.3 (continued) 

Corpus Christi Fort Worth Houston 

ERA Control Difference ERA Control Difference ERA Control Difference 

Outcome Group Group (Impact) Group Group (Impact) Group Group (Impact) 
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J Health coverage measures combine data from the survey employment section, health coverage section, income section, and administrative 
records on public assistance receipt. A person can be receiving both public and private health coverage. 
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worked at interview 33.4 35.7 -2.3 36.1 40.5 -4.4 34.8 33.5 
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Years 1-2, Impacts on TANF Receipt and Payments 
Texas 
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MDRC calculations from Texas administrative records. 
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NOTES: See Appendix B. 
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Sample size (total = 865) 146 133 111 114 




Appendix Table E.9 (continued) 

Corpus Christi Fort Worth Houston 

ERA Control Difference ERA Control Difference ERA Control Difference 

Outcome Group Group (Impact) Group Group (Impact) Group Group (Impact) 
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The Employment Retention and Advancement Project 
Appendix Table E.10 

Years 1-2, Impacts on Ul-Covered Employment and Earnings 
Corpus Christi and Fort Worth, Texas 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Years 1-2 








Ever employed (%) 


81.7 


80.7 


1.0 


Average quarterly employment (%) 


51.2 


48.5 


27** 


Employed 4 consecutive quarters (%) 


47.0 


43.2 


3.8 ** 


Earnings ($) 


9,147 


8,634 


513 


Earned over $20,000 (%) 


15.5 


13.1 


2.4 * 


Year 1 








Ever employed (%) 


71.6 


70.7 


0.9 


Average quarterly employment (%) 


51.4 


48.5 


2.9 * 


Employed 4 consecutive quarters (%) 


28.8 


25.9 


2.8 * 


Earnings ($) 


4,177 


3,919 


258 


Earned over $10,000 (%) 


13.5 


12.9 


0.6 


Year 2 








Ever employed (%) 


70.1 


66.6 


3.4 * 


Average quarterly employment (%) 


51.0 


48.5 


2.5 * 


Employed 4 consecutive quarters (%) 


30.9 


30.4 


0.5 


Earnings ($) 


4,970 


4,715 


255 


Earned over $10,000 (%) 


19.5 


18.1 


1.3 


Last auarter of Year 2 








Ever employed (%) 


51.6 


46.8 


4 8 ** 


Total earnings ($) 


1,316 


1,184 


132 * 


Earned $2,500 or more (%) 


22.4 


20.0 


2.4 


Earned between $500 and $2,499 (%) 


20.1 


19.9 


0.2 


Earned between $1 and $499 (%) 


9.1 


7.0 


2 2** 


Sample size (total = 2,470) 


1,232 


1,238 





SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the 
State of Texas. 



NOTES: See Appendix B. 

This table includes only employment and earnings in jobs covered by the Texas unemployment 
insurance (UI) program. It does not include employment outside Texas or in jobs not covered by UI 
(for example, "off-the-books" jobs, some agricultural jobs, and federal government jobs). 
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The Employment Retention and Advancement Project 
Appendix Table E.ll 

Years 1-2, Impacts on Ul-Covered Employment and Earnings 

Texas (All Sites) 

ERA Control Difference 

Outcome Group Group (Impact) 



Years 1-2 

Ever employed 3 (%) 

Average quarterly employment (%) 
Employed 4 consecutive quarters (%) 
Earnings (S) 

Earned over $20,000 (%) 

Year 1 

Ever employed (%) 

Average quarterly employment (%) 
Employed 4 consecutive quarters (%) 
Earnings ($) 

Earned over $10,000 (%) 

Year 2 

Ever employed (%) 

Average quarterly employment (%) 
Employed 4 consecutive quarters (%) 
Earnings ($) 

Earned over $10,000 (%) 

Last quarter of Year 2 

Ever employed (%) 

Total earnings ($) 

Earned $2,500 or more (%) 

Earned between $500 and $2,499 (%) 
Earned between $1 and $499 (%) 



77.8 


77.0 


0.8 


47.5 


45.8 


1.7 * 


42.8 


40.3 


2.6 * 


8,757 


8,510 


247 


15.1 


13.8 


1.4 


68.4 


67.6 


0.8 


47.6 


46.4 


1.2 


25.6 


24.7 


0.9 


4,006 


3,902 


104 


13.1 


13.3 


-0.2 


65.6 


62.8 


2.8 ** 


47.5 


45.3 


2.3 * 


28.6 


27.8 


0.7 


4,751 


4,608 


143 


18.5 


17.7 


0.8 


46.7 


44.6 


2.1 


1,217 


1,183 


33 


20.5 


20.4 


0.1 


18.4 


17.6 


0.7 


7.9 


6.6 


1.2 



Sample size (total = 4,288) 2,137 2,151 

SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the 
State of Texas. 

NOTES: See Appendix B. 

This table includes only employment and earnings in jobs covered by the Texas unemployment 
insurance (UI) program. It does not include employment outside Texas or in jobs not covered by UI 
(for example, "off-the-books" jobs, some agricultural jobs, and federal government jobs). 
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The Employment Retention and Advancement Project 
Appendix Table E.12 



Selected Characteristics of Texas ERA Sample Members 3 



Characteristic 


ERA 

Group 


Control 

Group 


Total 


Gender (%) 


Male 


3.1 


3.4 


3.3 


Female 


96.9 


96.6 


96.7 


Average age (years) 


28.4 


28.2 


28.3 


U.S. citizen 


Yes 


97.9 


97.8 


97.9 


No 


2.1 


2.2 


2.1 


Limited English 


Yes 


2.6 


2.2 


2.4 


No 


97.4 


97.8 


97.6 


Race/ethnicity 


White, non-Hispanic 


14.6 


14.4 


14.5 


Black, non-Hispanic 


47.7 


47.0 


47.3 


American Indian/ Alaska native 


0.1 


0.0 


0.1 


Hispanic 


36.6 


37.2 


36.9 


Asian/Pacific Islander 


0.3 


0.3 


0.3 


Other race 


0.7 


1.0 


0.9 


Education 


GED 


15.3 


15.5 


15.4 


High school diploma 


30.2 


27.1 


28.7 


Technical/ Associate's degree/2-year college 


3.2 


3.3 


3.2 


4 year (or more) college 


0.6 


0.9 


0.7 


None of the above 


50.7 


53.2 


52.0 


Current cash assistance status 


Applicant 


83.2 


83.2 


83.2 


Recipient 


16.8 


16.8 


16.8 


Registration status 


Mandatory 


75.7 


75.2 


75.5 


Exempt 


24.3 


24.8 


24.5 



(continued) 
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Appendix Table E.12 (continued) 



Characteristic 


ERA 

Group 


Control 

Group 


Total 


Total prior AFDC/TANF receipt (%) 


None 


35.9 


35.6 


35.7 


Less than 3 months 


7.4 


7.8 


7.6 


3 months or more and less than 2 years 


36.8 


38.3 


37.5 


2 years or more and less than 5 years 


11.5 


10.7 


11.1 


5 years or more and less than 10 years 


6.1 


5.6 


5.8 


10 years or more 


2.3 


2.0 


2.2 


Months employed in last 3 years 


Did not work 


15.1 


14.0 


14.5 


6 or less 


21.8 


20.9 


21.4 


7 to 12 


18.8 


20.5 


19.6 


13 to 24 


20.7 


19.9 


20.3 


More than 24 


23.6 


24.8 


24.2 


Type of employment in last 3 years (among those who worked) (%) 


Mostly part time 


24.2 


23.7 


23.9 


Mostly full time 


65.2 


64.1 


64.6 


Equal amounts part time and full time 


10.6 


12.2 


11.4 


Currently employed (%) 


Yes 


6.5 


7.2 


6.9 


No 


93.5 


92.8 


93.1 


Hours worked per week (among those currently employed) (%) 


10 or less 


10.4 


6.7 


8.5 


1 1 to 20 


28.0 


24.4 


26.2 


21 to 40 


41.6 


39.3 


40.4 


More than 40 


20.0 


29.6 


25.0 


Average hourly wage (among those currently employed) ($) 


6.73 


6.43 


6.57 


Number of children 


0 


0.9 


1.1 


1.0 


1 


39.9 


40.0 


39.9 


2 


28.2 


30.9 


29.6 


3 or more 


30.9 


28.0 


29.5 


Sample size (total = 4,288) 


2,137 


2,151 





SOURCE: Texas baseline information sheets. 



NOTE: Tests of statistical significance across the research groups were performed. No differences 
were statistically signficant. 

'This table includes single-parent ERA sample members only. Two-parent families are not 
included. 
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NOTES: See Append! 







The Employment Retention and Advancement Project 
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“Year 1” refers to Quarters 2 to 5. Quarter 1 is the quarter in which random assignment took place. 

Dollar averages include zero values for sample members who were not employed or were not receiving TANF or food stamps. 

Unless otherwise stated, results are for sample members randomly assigned from October 2000 to June 2002. 

This table includes only employment and earnings in jobs covered by the Texas unemployment insurance (UI) program. It does not include employment 
outside Texas or in jobs not covered by UI (for example, "off-the-books" jobs, some agricultural jobs, and federal government jobs). 




The Employment Retention and Advancement Project 
Appendix Table E.16 

Years 1-2, Impacts on Employment and Earnings Among Subgroups 
Defined by Ul-Covered Employment in Prior Year 
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Sample size (total = 1,221) 175 189 159 140 286 272 




Appendix Table E.16 (continued) 
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The Employment Retention and Advancement Project 

Appendix Table E.17 

Most Common Industries/Occupations Among the Currently Employed 





ERA 


Average 


Average 








Members 


Weekly 


Hourly 




Full-Time 




Employed 


Pay 


Pay 


Good Job 


Job 


Cluster 


(%) 


($) 


($) 


(%) 


(%) 


Job/oecupation 

Office and administrative support 


19.0 


305 


8.08 


48.0 


87.0 


Food preparation and serving-related 


16.0 


230 


6.63 


20.0 


78.0 


Sales and sales-related 


15.0 


209 


6.30 


6.0 


73.0 


Health care support 


11.0 


244 


7.81 


33.0 


58.0 


Personal care and service 


10.0 


110 


3.88 


6.0 


59.0 


Business/industrv 












Health care and social assistance 


27.0 


230 


6.85 


28.0 


67.0 


Accommodation and food services 
Retail trade of motor vehicles and 


17.0 


212 


6.09 


9.0 


79.0 


parts, furniture and home 
furnishings 


9.0 


218 


6.65 


10.0 


60.0 


Other services (except public 
administration) 

Retail trade of sport goods, hobbies, 


8.0 


231 


6.65 


23.0 


77.0 


books, music, or general 
merchandise 


5.0 


242 


7.45 


19.0 


81.0 
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-20% Report sample: October 2000 to June 2002 -$200 L Report sample: October 2000 to June 2002 




Appendix Figure E.l (continued) 
Houston 
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The Employment Retention and Advancement Project 
Appendix Figure E.2 



Estimated Eligibility for and Use of the ERA Stipend Among 100 TANF Applicants and 

Recipients in Fort Worth a 




SOURCES: MDRC calculations from UI administrative records from the State of Texas and ERA program 
tracking data. 

NOTES: a Among all ERA group members randomly assigned from October 2000 through January 2003. 

b The proportion of individuals who worked for four months and thereby completed the earnings disregard 
period was estimated using UI records. Individuals were determined to have worked for four months if they 
were employed for two consecutive quarters within a year and had total earnings of more than $2,400 during 
this period (this is the equivalent of working 20 hours per week for four months at $7 per hour). 
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The Employment Retention and Advancement Project 
Appendix Figure E.3 

Estimated Eligibility for and Use of the ERA Stipend Among 100 TANF Applicants and 

Recipients in Houston 3 




SOURCES: MDRC calculations from UI administrative records from the State of Texas and ERA 
program tracking data. 

NOTES: “Among all ERA group members randomly assigned from October 2000 through January 
2003. 

b The proportion of individuals who worked for four months and thereby completed the earnings 
disregard period was estimated using UI records. Individuals were detennined to have worked for 
four months if they were employed for two consecutive quarters within a year and had total earnings 
of more than $2,400 during this period (this is the equivalent of working 20 hours per week for four 
months at $7 per hour). 
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The Employment Retention and Advancement Project 
Appendix Figure E.4 

Impacts on Program Participation for Key Subgroups in Corpus Christi 

Texas 



Percentage points 



0 5 10 15 20 25 



Earned $5,000 or 
more in prior 
year 
(N = 49) 



22.3 




Earned less than 
$5,000 in prior 
year 

(N= 116) 




13.7 



Unemployed in 
prior year 

(N = 77) 




□ Received help with 
retention/advancement 

■ Received help with job 
preparation 

□ Had any contacts with case 
manager/employment program 
since random assignment 



Employed in 
prior year 

(N = 213) 




SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 
NOTES: See Appendix D. 
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The Employment Retention and Advancement Project 
Appendix Figure E.5 

Impacts on Program Participation for Key Subgroups in Fort Worth 

Texas 



Percentage points 



-40 -30 -20 -10 0 10 20 30 40 



Earned $5,000 or 
more in prior 
year 
(N =42) 




Earned less than 
$5,000 in prior 
year 

(N = 70) 



Unemployed in 
prior year 

(N = 49) 




□ Received help with 
retention/advancement 

■ Received help with job 
preparation 

0 Had any contacts with case 
manager/employment program 
since random assignment 



Employed in 
prior year 

(N= 139) 




SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 
NOTES: See Appendix D. 
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The Employment Retention and Advancement Project 
Appendix Figure E.6 

Impacts on Program Participation for Key Subgroups in Houston 

Texas 



Percentage points 



-20 -10 0 10 20 30 40 50 



Earned $5,000 
or more in prior 
year 

(N = 45) 



16.3 




47.1 



Earned less 
than $5,000 in 
prior year 

(N= 135) 



Unemployed in 
prior year 

(N = 78) 





□ Received help with 
retention/advancement 

■ Received help with job preparation 



□ Had any contacts with case 
manager/employment program since 
random assignment 



Employed in 
prior year 

(N = 2 1 9) 




SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 
NOTES: See Appendix D. 
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The Employment Retention and Advancement Project 
Appendix Figure E.7 

Percentage Employed According to UI Records and Receiving the ERA Stipend 

Fort Worth 
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SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the State of Texas. 
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NOTES: See Appendix B. 
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The Employment Retention and Advancement Project 

Appendix Figure E.9 

Impacts on Ul-Covered Earnings and Employment for the Report Sample and Early Cohort Over Time 

Texas 

Corpus Christi 




(continued) 




Appendix Figure E.9 (continued) 

Houston 
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SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the State of Texas. 



NOTES: See Appendix B. 




Appendix F 

Texas ERA 12-Month Survey Response Analysis 




Corpus Christi Survey Response Analysis 

This section of the appendix assesses the reliability of impact results from the one-year 
survey of the Employment Retention and Advancement (ERA) evaluation in Corpus Christi, 
Texas. It examines whether the impacts for respondents to the ERA 12-Month Survey can be gen- 
eralized to the impacts for the report sample in Corpus Christi. After introducing how the survey 
sample was selected, it discusses the response rates for survey sample members and for sample 
members in the research groups. It also examines the differences in background characteristics 
between survey respondents and nonrespondents and between research groups among survey re- 
spondents. Finally, this section compares the impacts across the survey sample and the report 
sample in Corpus Christi on measures created from the administrative records. 



Survey Selection in Corpus Christi 

The report sample in Corpus Christi includes 1,309 sample members who were ran- 
domly assigned to the ERA group (N = 656) or to the control group (N = 653) from October 
2000 through June 2002. 

A two-step process was used to select the ERA 12-Month Survey sample in Corpus 
Christi. First, the survey-eligible sample was selected. This sample includes 357 sample members 
who met the survey eligibility criteria. Research sample members who were randomly assigned 
from January through June 2002, who were age 1 8 or older, who were in a single-parent family, 
and who spoke either English or Spanish were eligible for the survey. It is particularly notable that 
the survey sample was drawn from a narrow 6-month segment of the sample intake period; this 
raises concerns about the generalizability of results to the full report sample (which covered a 26- 
month period in Corpus Christi). 

In Corpus Christi, all sample members who were eligible for the survey were selected 
to be interviewed. The interview sample is referred to as th q fielded survey sample and is split 
equally between members of the ERA group (N = 1 80) and members of the control group (N = 
177). Box F.l describes the key analysis samples in Corpus Christi. 



Survey Response Rates in Corpus Christi 

Sample members who were interviewed for the ERA 12-Month Survey are referred to 
as survey respondents, while sample members who were not interviewed are known as nonre- 
spondents. 

Although the overall response rate in Corpus Christi was 82 percent, nonresponse bias 
may occur whenever the response rate is below 100 percent, because there may be differences 
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in the average background characteristics of the respondent sample and the fielded sample. The 
response rate was slightly lower among the ERA group (78 percent) than among the control 
group (85 percent). 1 

The primary reasons that some sample members in Corpus Christi were not interviewed 
(N = 67) are that they could not be located (N = 54), they refused to be interviewed (N = 9), 
they were incapacitated (N = 3), or they were ineligible to be surveyed (N = 1). 



Box F.1 

Key Analysis Samples in Corpus Christi 

Research sample. Sample members who were randomly assigned during the sample intake 
period, which started in October 2000 and ended in December 2002. 

Report sample. Single parents who were randomly assigned from October 2000 through 
June 2002. 

Survey-eligible sample. Sample members in the research sample who were randomly as- 
signed from January through June 2002 and who met the criteria for inclusion in the survey. 

Fielded sample. Sample members who were chosen from the survey-eligible sample to be 
interviewed for the survey. In the case of Corpus Christi, the fielded sample and the survey- 
eligible sample are the same. 

Respondent sample. Sample members in the fielded sample who completed the ERA 12- 
Month Survey. 

Nonrespondent sample. Sample members in the fielded sample who were not interviewed 
because they were not located or they refused to be interviewed or because of other reasons. 



Comparison of Respondents and Nonrespondents Within the 
Survey Sample: Corpus Christi 

This section examines whether there are differences in pre-random assignment charac- 
teristics between respondents and nonrespondents within the survey-eligible sample in Corpus 
Christi. In order to examine differences between those who completed the survey and those who 
did not, MDRC created an indicator of survey response and related the indicator to pre-random 
assignment characteristics in a multivariate analysis. 



*OMB stipulates a response rate of 80 percent. 
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Appendix Table F.l shows estimated regression coefficient for the probability of being 
a respondent in Corpus Christi, based on background characteristics. The first column of the 
table provides the parameter estimates that indicate the effect of each variable on the probability 
of completing the survey. The stars (asterisks) and p-values show whether the relationships are 
statistically significant. 



The Employment Retention and Advancement Project 
Appendix Table F.l 



Estimated Regression Coefficients for the Probability of Being a Respondent 
on the ERA 12-Month Survey, Corpus Christi 





Parameter 

Estimate 


Survey Sample 

P -Value 


ERA group 


-0.054 


0.199 


Age of the youngest child 


-0.001 


0.889 


Number of children 


-0.016 


0.442 


Black, non-Hispanic 


0.074 


0.593 


White 


0.102 


0.350 


Hispanic 


0.132 


0.209 


No high school diploma or GED 


-0.091 ** 


0.042 


Employed in the quarter prior to random assignment 


-0.003 


0.953 


Female 


-0.122 


0.145 


Month of sample intake 


0.001 


0.956 


21-30 years of age 


0.021 


0.772 


3 1 -40 years of age 


0.085 


0.353 


4 1 years old and over 


0.105 


0.304 


Employed in the prior year 


0.055 


0.422 


Received food stamps in the prior year 


0.142 *** 


0.004 


Earnings in the prior 3 years 


0.000 


0.413 


Number of quarters employed in prior 3 years 


0.001 


0.947 


R-square (0.081) 






F-statistic (1.75) 






P-value of F-statistic (0.033) 






Sample size 


357 





SOURCE: MDRC calculations from administrative records from the State of Texas. 

NOTE: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 
* = 10 percent. 



Appendix Table F.l shows that survey respondents and nonrespondents in Corpus 
Christi were similar at random assignment: Most measures of background characteristics are 
unable to differentiate between them. However, food stamp receipt in the year prior to random 
assignment and not having a high school diploma or General Educational Development (GED) 
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certificate are both statistically significant characteristics in predicting whether someone would 
complete a survey in Corpus Christi. Survey respondents were more likely to have received 
food stamps in the year prior to random assignment and to have a high school diploma or GED. 
It is not surprising to see that food stamp receipt in the year prior to random assigmnent is a 
good predictor of survey response. Other recent response analyses have found this relationship 
as well. This is likely attributable to better and more updated contact information for these sam- 
ple members. It is less clear why having no high school diploma or GED is statistically signifi- 
cant in predicting survey completion. 

At the bottom of Appendix Table F.l, the F-statistic and the p-value of the F-statistic 
show that the overall difference between survey respondents and nonrespondents in Corpus 
Christi is statistically significant (p-value = 0.003). Thus, caution should be exercised in gener- 
alizing these results to the report sample. 



Comparison of the Research Groups in the Survey Respondent 
Sample: Corpus Christi 

The random assignment design essentially eliminates the possibility of selection bias. 
However, the risk of differences between the background characteristics of ERA and control 
group members is reintroduced due to the survey sampling and response process. Specifically, if 
different types of ERA group members respond (compared with control group members), the 
integrity of the experiment would be compromised for the survey analysis. If this is true to a 
large extent, the ability to generalize from the respondent sample may be affected. 

Appendix Table F.2 shows whether there may be any statistically significant differ- 
ences in background characteristics between ERA group and control group respondents in Cor- 
pus Christi. Overall, there are few differences between them. The main differences are that ERA 
group respondents at random assigmnent were younger, had lower earnings, and were more 
likely to have received food stamps in the prior year. 



Comparison of Survey Respondents with the Fielded Sample and 
the Report Sample: Corpus Christi 

This section examines whether impacts among survey respondents in Corpus Christi 
can be generalized to the fielded sample and the report sample and whether any cohort effects 
may have been introduced. 2 



2 The impacts presented in this appendix are statistically significant unless otherwise noted. 
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The Employment Retention and Advancement Project 
Appendix Table F.2 



Background Characteristics of Survey Respondents, Corpus Christi 



Variable 


ERA 

Group 


Control 

Group 


Female (%) 


5.7 


8.1 


Race/ethnicity (%) 






Hispanic 


73.8 


74.3 


Black 


5.7 


4.7 


White 


19.1 


17.6 


Other 


1.4 


3.4 


Age (%) 






20 or younger 


12.1 


8.7 * 


21-30 


53.9 


47.7 * 


31-40 


26.2 


25.5 * 


4 1 or older 


7.8 


18.1 * 


Average age (years) 


28 


30 * 


High school diploma (%) 


48.6 


57.0 


Employed during the quarter prior to random assigmnent (%) 


49.6 


47.0 


Employed during the year prior to random assigmnent (%) 


75.9 


71.1 


Number of quarters employed in the prior 3 years (%) 


6.3 


6.5 


Earnings in the 3 years prior to random assigmnent ($) 


11,629 


15,571 ** 


Number of children (%) 






0 


0.7 


0.7 


1 


43.3 


49.7 


2 


27.0 


28.2 


More than 3 


29.1 


21.5 


Average number of children 


2.0 


1.8 


Age of youngest child (%) 






Under 3 years 


55.7 


49.3 


3-5 years 


22.1 


20.3 


6 years and older 


22.1 


30.4 


AFDC receipt history (%) 






Never 


46.1 


43.6 


Less than 3 months 


4.3 


6.0 


3 months or more and less than 2 years 


34.8 


30.9 


2 years or more and less than 5 years 


9.9 


10.7 


5 years or more and less than 10 years 


3.5 


7.4 


10 years or more 


1.4 


1.3 


Received food stamps in prior year (%) 


80.1 


70.5 * 


Sample size (total = 290) 


141 


149 



SOURCES: Texas background information form and administative records from the State of 
Texas. 

NOTES: Results are for sample members randomly assigned from January through June 2002. 
Chi-square (categorical) and two-tailed T (continous) tests were used to assess the differences 
in characteristics across research groups. Statistical significance levels are indicated as 
follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Appendix Table F.3 shows the adjusted means and impacts on employment and welfare 
outcomes for Corpus Christi’s full sample, fielded sample, and respondent sample.’ With some 
exceptions, the impacts are consistent across the samples. 4 In some cases, smaller impacts are 
statistically significant among the report sample, due to the larger sample size. Appendix Table 
F.3 shows that the impact on earnings is larger among the report sample than the respondent 
sample, for reasons that are unclear. 



Conclusions About Corpus Christi 

Due to the narrow cohort of months from which Corpus Christi’s survey sample was 
selected — as well as some evidence of response bias — caution should be exercised when 
generalizing the results among the survey sample to the broader report sample. In particular, this 
response analysis found that survey respondents were more likely than nonrespondents to have 
received food stamps in the year prior to random assigmnent and were more likely to have a 
high school diploma or GED. Furthermore, respondents in the ERA group were younger than 
their control group counterparts and were more likely to have received food stamps in the year 
prior to random assigmnent. Finally, there were some small differences across the three samples 
in the pattern of impacts on measures of administrative records. 



’All the impacts are regression-adjusted within each sample, to control for differences in background char- 
acteristics, prior earnings, prior employment, prior public assistance receipt, location or residence, and period 
of sample intake. 

4 The consistency in impacts for the respondent, fielded, and report samples refers to the direction and 
magnitude of the impacts, not to whether they are all statistically significant. In fact, the impacts on average 
quarterly employment, employment over four consecutive quarters, number of quarters employed, and per- 
centage ever receiving TANF are statistically significant among sample members in the report sample but are 
not significant among sample members in the respondent and the fielded samples. 
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The Employment Retention and Advancement Project 
Appendix Table F.3 



Comparison of Impacts for the Full, 

Fielded, and Respondent Samples in Corpus Christi 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Ouarters 2-5 

Ever employed (%) 


Full sample 


72.7 


73.7 


-1.0 


F ielded sample 


73.1 


71.8 


1.3 


Respondent sample 


75.3 


73.2 


2.1 


Average quarterly employment (%) 


Full sample 


52.4 


49.5 


2.9 * 


F ielded sample 


54.8 


49.5 


5.3 


Respondent sample 


58.8 


53.2 


5.6 


Employed 4 consecutive quarters (%) 


Full sample 


29.9 


26.1 


3.8 * 


F ielded sample 


32.9 


27.9 


5.0 


Respondent sample 


36.6 


33.2 


3.3 


Number of quarters employed 


Full sample 


2.1 


2.0 


0.1 * 


F ielded sample 


2.2 


2.0 


0.2 


Respondent sample 


2.4 


2.1 


0.2 


Earnings ($) 


Full sample 


3,982 


3,567 


415 * 


F ielded sample 


4,229 


3,847 


382 


Respondent sample 


4,518 


4,353 


165 


Ever received TANF (%) 


Full sample 


81.9 


78.0 


4.0 ** 


F ielded sample 


83.7 


76.6 


7.1 


Respondent sample 


81.8 


76.6 


5.3 


Amount of food stamps received ($) 


Full sample 


2,745 


2,820 


-75 


F ielded sample 


2,632 


2,743 


-111 


Respondent sample 


2,644 


2,789 


-145 


Total measured income ($) 


Full sample 


7,751 


7,376 


375 * 


F ielded sample 


7,876 


7,580 


296 


Respondent sample 


8,184 


8,115 


70 



SOURCE: MDRC calculations from administrative records from the State of Texas. 



NOTES: The full sample includes 1,723 sample members; experimental: 868; control: 
855. The fielded sample includes 357 sample members; experimental: 180; control: 

177. The respondent sample includes 290 sample members; experimental: 141; control: 
149. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 
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Fort Worth Survey Response Analysis 

This section of the appendix assesses the reliability of impact results from the one-year 
survey of the Employment Retention and Advancement (ERA) evaluation in Fort Worth, Texas. 
It examines whether the impacts for respondents to the ERA 12-Month Survey can be general- 
ized to the impacts for the report sample in Fort Worth. After introducing how the survey sam- 
ple was selected, it discusses the response rates for survey sample members and for sample 
members in the research groups. It also examines the differences in background characteristics 
between survey respondents and nonrespondents and between research groups among survey 
respondents. Finally, this section compares the impacts across the survey sample and the report 
sample on measures created from the administrative records. The survey sample in Fort Worth 
is the smallest of the three Texas sites and is drawn from a cohort that does not overlap at all 
with the report sample cohort. For this reason, it is not advisable to make inferences about Fort 
Worth’s broader report sample on the basis of the survey sample’s impacts. 



Survey Selection in Fort Worth 

The report sample in Fort Worth includes 1,163 sample members who were randomly 
assigned to the ERA group (N = 577) or to the control group (N = 586) from October 2000 
through June 2002. 

As in the other sites, a two-step process was used to select the ERA 12-Month Survey 
sample in Fort Worth. First, the survey-eligible sample was selected. This included 250 sample 
members who met the survey eligibility criteria. Research sample members who were randomly 
assigned from September through December 2002, who were age 18 or older, who were in a 
single-parent family, and who spoke either English or Spanish were eligible for the survey. It is 
particularly notable that the survey sample was drawn from a very narrow 3-month segment of 
the sample intake period that does not overlap at all with the report sample cohort; this raises 
substantial concerns about the generalizability of results to the full report sample (which covers 
a 26-month period in Fort Worth). 

In Fort Worth, all sample members eligible for the survey were selected to be inter- 
viewed. The interview sample is referred to as the fielded survey sample and is split fairly 
evenly between members of the ERA group (N = 123) and members of the control group (N = 
127). Box F.2 describes the key analysis samples in Fort Worth. 
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Box F.2 



Key Analysis Samples in Fort Worth 



Research sample. Sample members who were randomly assigned during the sample intake 
period, which started in October 2000 and ended in December 2002. 

Report sample. Single parents who were randomly assigned from October 2000 through 
June 2002. 

Survey-eligible sample. Sample members in the research sample who were randomly as- 
signed from September through December 2002 and who met the criteria for inclusion in the 
survey. 

Fielded sample. Sample members who were chosen from the survey-eligible sample to be 
interviewed for the survey. 

Respondent sample. Sample members in the fielded sample who completed the ERA 12- 
Month Survey. 

Nonrespondent sample. Sample members in the fielded sample who were not interviewed 
because they were not located or they refused to be interviewed or because of other reasons. 



Survey Response Rates in Fort Worth 

Sample members who were interviewed for the ERA 12-Month Survey are referred to 
as survey respondents, while sample members who were not interviewed are known as non- 
respondents. 

Although the overall response rate in Fort Worth was about 75 percent (N = 188), this is 
slightly below the OMB standard for response rates (80 percent), and thus the risk of nonre- 
sponse bias is somewhat higher. Roughly equal proportions of ERA group members (76 per- 
cent) and control group members (75 percent) responded to the survey. 5 

The primary reasons that some sample members in Fort Worth were not interviewed (N 
= 62) are that they could not be located (N = 35), they refused to be interviewed (N = 14), or 
they were incapacitated (N = 4). 



Comparison of Respondents and Nonrespondents Within the 
Survey Sample: Fort Worth 

This section examines whether there are differences in pre-random assignment charac- 
teristics between respondents and nonrespondents within the survey-eligible sample in Fort 

5 OMB stipulates a response rate of 80 percent. 
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Worth. In order to examine differences between those who completed the survey and those who 
did not, MDRC created an indicator of survey response and related the indicator to pre-random 
assignment characteristics in a multivariate analysis. 

Appendix Table F.4 shows estimated regression coefficient for the probability of being 
a respondent in Fort Worth, based on background characteristics. The first column of the table 
provides the parameter estimates that indicate the effect of each variable on the probability of 
completing the survey. The star (asterisk) and p-value show the statistical significance of this 
relationship. 

Appendix Table F.4 shows that survey respondents and nonrespondents in Fort Worth 
were similar at random assignment: No measures of background characteristics were able to 
differentiate between them. However, whether a sample member had received food stamps 
prior to random assignment comes close to being a statistically significant predictor of survey 
response (p-value = 0.101). At the bottom of Appendix Table F.4, the F-statistic and the p-value 
of the F-statistic show that the overall difference between survey respondents and nonrespon- 
dents is not statistically significant. 



Comparison of the Research Groups in the Survey Respondent 
Sample: Fort Worth 

The random assignment design essentially eliminates the possibility of selection bias. 
However, the risk of differences between the background characteristics of ERA and control 
group members is reintroduced due to the survey sampling and response process. Specifically, if 
different types of ERA group members respond (compared with control group members), the 
integrity of the experiment would be compromised for the survey analysis. If this is true to a 
large extent, the ability to generalize from the respondent sample may be affected. 

Appendix Table F.5 shows whether there may be any statistically significant differ- 
ences in background characteristics between ERA group and control group respondents in Fort 
Worth. There are no statistically significant differences in background characteristics between 
ERA group survey respondents and control group survey respondents. In other words, the integ- 
rity of the experiment is preserved. For example, among survey respondents, ERA group mem- 
bers were employed, on average, 7.1 quarters in the three years prior to random assigmnent, 
which is very close to the control group’s average of 6.9 quarters. 
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The Employment Retention and Advancement Project 
Appendix T able F.4 



Estimated Regression Coefficients for the Probability of Being a Respondent 
on the ERA 12-Month Survey, Fort Worth 



Survey Sample 

Parameter 

Estimate P -Value 


ERA group 


0.003 


0.955 


Age of the youngest child 


-0.004 


0.667 


Number of children 


0.041 


0.181 


Black, non-Hispanic 


-0.172 


0.359 


White 


-0.245 


0.185 


Hispanic 


-0.180 


0.346 


No high school diploma or GED 


0.047 


0.434 


Employed in the quarter prior to random assignment 


-0.054 


0.498 


Female 


0.040 


0.772 


Month of sample intake 


0.023 


0.387 


21-30 years of age 


-0.116 


0.251 


3 1 -40 years of age 


- 0.111 


0.381 


41 years old and over 


-0.004 


0.981 


Employed in the prior year 


-0.052 


0.593 


Received food stamps in the prior year 


0.101 


0.101 


Earnings in the prior 3 years 


0.000 


0.813 


Number of quarters employed in prior three years 


0.019 


0.169 


R-square (0.066) 






F-statistic (0.96) 






P-value of F-statistic (0.499) 






Sample size 


250 





SOURCE: MDRC calculations from administrative records from the State of Texas. 

NOTE: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 
* = 10 percent. 



Comparison of Survey Respondents with the Fielded Sample and 
the Report Sample: Fort Worth 

This section examines whether impacts among survey respondents in Fort Worth can be 
generalized to the fielded sample and the report sample and whether any cohort effects may 
have been introduced . 6 



6 The impacts presented in this appendix are statistically significant unless otherwise noted. 
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The Employment Retention and Advancement Project 
Appendix Table F.5 



Background Characteristics of Survey Respondents, Fort Worth 



Variable 


ERA 

Group 


Control 

Group 


Female (%) 


95.7 


95.8 


Race/ethnicity (%) 


Hispanic 


15.2 


15.6 


Black 


64.1 


61.5 


White 


19.6 


19.8 


Other 


1.1 


3.1 


Age (%) 


20 or younger 


13.0 


10.4 


21-30 


46.7 


52.1 


31-40 


31.5 


26.0 


41 or older 


8.7 


11.5 


Average age years 


28 


29 


High school diploma (%) 


54.3 


57.3 


Employed during the quarter prior to random assignment (%) 


56.5 


50.0 


Employed during the year prior to random assignment (%) 


76.1 


71.9 


Number of quarters employed in the prior 3 years (%) 


7.1 


6.9 


Earnings in the 3 years prior to random assigmnent ($) 


19,242 


18,914 


Number of children (%) 


1 


44.6 


37.5 


2 


31.5 


33.3 


More than 3 


23.9 


29.2 


Average number of children 


1.9 


2.0 


Age of youngest child (%) 


Under 3 years 


55.4 


54.2 


3-5 years 


15.2 


16.7 


6 years and older 


29.3 


29.2 


AFDC receipt history (%) 


Never 


41.3 


41.7 


Less than 3 months 


2.2 


7.3 


3 months or more and less than 2 years 


54.3 


42.7 


2 years or more and less than 5 years 


1.1 


5.2 


5 years or more and less than 10 years 


1.1 


3.1 


Received food stamps in prior year (%) 


70.7 


66.7 


Sample size (total = 188) 


92 


96 



SOURCES: Texas background information form and administative records from the State of 
Texas. 



NOTE: Results are for sample members and randomly assigned from September through 
December 2002. Chi-square (categorical) and two-tailed T (continous) tests were used to 
assess the differences in characteristics across research groups. Statistical significance levels 
are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Appendix Table F.6 shows the adjusted means and impacts on employment and welfare 
outcomes for Fort Worth’s report sample, fielded sample, and the respondent sample. 7 The 
comparison shows that the impacts among the fielded sample and among respondent sample are 
inconsistent with the impacts among the report sample on most measures of earnings, employ- 
ment, and food stamp receipt. This is not surprising, given that the respondent and fielded sam- 
ples do not at all overlap with the report sample. For example, among the report sample, ERA 
group members were 2.3 percentage points more likely to have been employed during Year 1 
than control group members. Among the respondent sample, ERA group members were nearly 
12 percentage points less likely to have been employed than control group members. There are 
large discrepancies on several other measures. Notably, ERA had a large impact on food stamp 
receipt among the respondent sample, which was not at all evident among the report sample. 



Conclusions About Fort Worth 

Due to the narrow cohort of months from which Fort Worth’s survey sample was se- 
lected — as well as the substantially different pattern of impacts among the survey respondent 
sample — great caution should be exercised when generalizing the results among the survey 
sample to the broader report sample. While the differences in background characteristics are not 
large, the impacts, in some cases, are qualitatively different across subsamples. This is likely 
because the survey sample does not overlap with the report sample. 



7 A11 the impacts are regression-adjusted within each sample to control for differences in background char- 
acteristics, prior earnings, prior employment, prior public assistance receipt, location or residence, and period 
of sample intake. 
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The Employment Retention and Advancement Project 
Appendix Table F.6 



Comparison of Impacts for the Report, 
Fielded, and Respondent Samples in Fort Worth 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Ouarters 2-5 

Ever employed (%) 


Report sample 


69.4 


67.1 


2.3 


Fielded sample 


64.6 


72.8 


-8.2 


Respondent sample 


63.7 


75.4 


-11.6 * 


Average quarterly employment (%) 


Report sample 


49.1 


47.1 


2.0 


Fielded sample 


41.2 


48.4 


-7.2 


Respondent sample 


40.6 


48.8 


-8.2 


Employed 4 consecutive quarters (%) 


Report sample 


26.4 


25.4 


1.0 


Fielded sample 


15.0 


28.8 


-13.8 *** 


Respondent sample 


15.4 


27.9 


-12.5 ** 


Number of quarters employed 


Report sample 


2.0 


1.9 


0.1 


Fielded sample 


1.7 


1.9 


-0.3 


Respondent sample 


1.6 


2.0 


-0.3 


Earnings ($) 


Report sample 


4,486 


4,248 


238 


Fielded sample 


3,540 


4,446 


-906 


Respondent sample 


3,583 


4,479 


-896 


Ever received TANF (%) 


Report sample 


80.8 


81.1 


-0.3 


Fielded sample 


76.3 


78.1 


-1.8 


Respondent sample 


81.1 


77.5 


3.6 


Amount of food stamps received ($) 


Report sample 


2,698 


2,672 


26 


Fielded sample 


3,142 


2,354 


788 *** 


Respondent sample 


3,298 


2,577 


721 *** 


Total measured income ($) 


Report sample 


8,469 


8,113 


355 


Fielded sample 


7,906 


7,771 


134 


Respondent sample 


8,194 


8,054 


140 



SOURCE: MDRC calculations from administrative records from the State of Texas. 



NOTE: The report sample includes 1,163 sample members; experimental: 577; control: 
586. The fielded sample includes 250 sample members; experimental: 123; control: 

127. The respondent sample includes 188 sample members; experimental: 92; control: 
96. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 
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Houston Survey Response Analysis 

This section of the appendix assesses the reliability of impact results from the one-year 
survey of the Employment Retention and Advancement (ERA) evaluation in Houston, Texas. It 
examines whether the impacts for respondents to the ERA 12-Month Survey can be generalized 
to the impacts for the report sample in Houston. After introducing how the survey sample was 
selected, it discusses the response rates for survey sample members and for sample members in 
the research groups. It also examines the differences in background characteristics between sur- 
vey respondents and nonrespondents and between research groups among survey respondents. 
Finally, this section compares the impacts across the survey sample and the report sample in 
Houston on measures created from the administrative records. 



Survey Selection in Houston 

The report sample in Houston includes 1,816 sample members who were randomly as- 
signed to the ERA group (N = 904) or to the control group (N = 912) from March 2001 through 
June 2002. 

As in the other sites, a two-step process was used to select the ERA 12-Month Survey 
sample in Houston. First, the survey-eligible sample was selected. This sample includes 501 
sample members who met the survey eligibility criteria. Research sample members who were 
randomly assigned from January through June 2002, who were age 18 or older, who were in a 
single-parent family, and who spoke either English or Spanish were eligible for the survey. It is 
particularly notable that the survey sample was drawn from a narrow 6-month slice of the sam- 
ple intake period; this raises concerns about the generalizability of results to the full report sam- 
ple (which covered a 16-month period in Houston). 

In Houston, 370 out of the 501 sample members eligible for the survey were selected to 
be interviewed. The interview sample is referred to as the fielded survey sample and is split 
fairly evenly between members of the ERA group (N = 1 87) and members of the control group 
(N = 183). Box F.3 describes the key analysis samples in Houston. 
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Box F.3 



Key Analysis Samples in Houston 



Research sample. Sample members who were randomly assigned during the sample intake 
period, which started in March 2001 and ended in December 2002. 

Report sample. Single parents who were randomly assigned from March 2001 through June 
2002 . 

Survey-eligible sample. Sample members in the research sample who were randomly as- 
signed from January through June 2002 and who met the criteria for inclusion in the survey. 

Fielded sample. Sample members who were chosen from the survey-eligible sample to be 
interviewed for the survey. 

Respondent sample. Sample members in the fielded sample who completed the ERA 12- 
Month Survey. 

Nonrespondent sample. Sample members in the fielded sample who were not interviewed 
because they were not located or they refused to be interviewed or because of other reasons. 



Survey Response Rates in Houston 

Sample members who were interviewed for the ERA 12-Month Survey are referred to 
as survey respondents, while sample members who were not interviewed are known as nonre- 
spondents. 

Although the overall response rate in Houston was approximately 80 percent (the OMB 
standard), nonresponse bias may occur whenever the response rate is below 100 percent, be- 
cause there may be differences in the average background characteristics of the respondent 
sample and the fielded sample. 

The primary reasons that some sample members in Houston were not interviewed (N = 
53) are that they could not be located (N = 54), they refused to be interviewed (N = 1 1), or they 
were incapacitated (N = 3). 



Comparison of Respondents and Nonrespondents Within the 
Survey Sample: Houston 

This section examines whether there are differences in pre-random assignment charac- 
teristics between respondents and nonrespondents within the survey-eligible sample in Houston. 
In order to examine differences between those who completed the survey and those who did 
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not, MDRC created an indicator of survey response and related the indicator to pre-random as- 
signment characteristics in a multivariate analysis. 

Appendix Table F.7 shows estimated regression coefficient for the probability of being 
a respondent in Houston, based on background characteristics. The first column of the table 
provides the parameter estimates that indicates the effect of each variable on the probability of 
completing the survey. The stars (asterisks) and p-values indicate whether the relationships are 
statistically significant. 



The Employment Retention and Advancement Project 
Appendix T able F.7 



Estimated Regression Coefficients for the Probability of Being a Respondent 
on the ERA 12-Month Survey, Houston 



Survey Sample 

Parameter 

Estimate P-Value 


ERA group 


-0.010 


0.820 


Age of the youngest child 


-0.001 


0.937 


Number of children 


0.010 


0.605 


Black, non-Hispanic 


0.103 


0.426 


White 


0.038 


0.789 


Hispanic 


0.118 


0.377 


No high school diploma or GED 


-0.002 


0.960 


Employed in the quarter prior to random assignment 


0.094 * 


0.082 


Female 


-0.096 


0.422 


Month of sample intake 


-0.004 


0.754 


21-30 years of age 


-0.124 


0.123 


3 1 -40 years of age 


-0.129 


0.184 


41 years old and over 


-0.152 


0.226 


Employed in the prior year 


-0.044 


0.526 


Received food stamps in the prior year 


0.045 


0.381 


Earnings in the prior 3 years 


0.000 


0.604 


Number of quarters employed in prior 3 years 


0.003 


0.771 


R-square (0.027) 






F-statistic (0.57) 






P-value of F-statistic (0.91 1) 






Sample size 


370 





SOURCE: MDRC calculations from administrative records from the State of Texas. 



NOTE: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; 
* = 10 percent. 
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Appendix Table F.7 shows that survey respondents and nonrespondents in Houston 
were similar at random assigmnent: Most measures of background characteristics are unable to 
differentiate between them. Employment in the quarter prior to random assignment is the only 
statistically significant characteristic in predicting whether someone would complete a survey in 
Houston. Survey respondents were more likely than nonrespondents to be employed in the 
quarter prior to random assigmnent. 

At the bottom of Appendix Table F.7, the F-statistic and the p-value of the F-statistic 
show that the full set of background characteristics is not able statistically to distinguish be- 
tween survey respondents and nonrespondents. 



Comparison of the Research Groups in the Survey Respondent 
Sample: Houston 

The random assignment design essentially eliminates the possibility of selection bias. 
However, the risk of differences between the background characteristics of ERA and control 
group members is reintroduced due to the survey sampling and response process. Specifically, if 
different types of ERA group members respond (compared with control group members), the 
integrity of the experiment would be compromised for the survey analysis. If this is true to a 
large extent, the ability to generalize from the respondent sample may be affected. 

Appendix Table F.8 shows whether there may be any statistically significant differ- 
ences in background characteristics between ERA group and control group respondents in 
Houston. There are no differences between them. In other words, the integrity of the experiment 
is preserved. 
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The Employment Retention and Advancement Project 
Appendix Table F.8 



Background Characteristics of Survey Respondents, Houston 



Variable 


ERA 

Group 


Control 

Group 


Female (%) 


96.7 


95.2 


Race/ethnicity (%) 


Hispanic 


28.7 


29.0 


Black 


61.3 


63.8 


White 


9.3 


7.2 


Other 


0.7 


0.0 


Age (%) 


20 or younger 


10.7 


10.2 


21-30 


54.7 


59.2 


31-40 


25.3 


21.8 


41 or older 


9.3 


8.8 


Average age (years) 


28 


28 


High school diploma (%) 


41.9 


42.8 


Employed during the quarter prior to random assignment (%) 


44.0 


47.6 


Employed during the year prior to random assignment (%) 


70.7 


76.9 


Number of quarters employed in the prior 3 years (%) 


6.1 


6.3 


Earnings in the 3 years prior to random assignment ($) 


13,633 


14,392 


Number of children (%) 


0 


0.0 


2.1 


1 


40.0 


34.9 


2 


26.0 


32.2 


More than 3 


34.0 


30.8 


Average number of children 


2.1 


2.1 


Age of youngest child (%) 


Under 3 years 


55.3 


60.1 


3-5 years 


16.7 


19.6 


6 years and older 


28.0 


20.3 


AFDC receipt history (%) 


Never 


45.5 


44.0 


Less than 3 months 


10.3 


9.9 


3 months or more and less than 2 years 


32.4 


31.9 


2 years or more and less than 5 years 


4.8 


9.9 


5 years or more and less than 10 years 


4.8 


2.1 


10 years or more 


2.1 


2.1 


Received food stamps in prior year (%) 


77.3 


72.8 


Sample size (total = 297) 


150 


147 



SOURCES: Texas background information form and administative records from the State of 
Texas. 



NOTES: Results are for sample members and randomly assigned from January through June 
2002. Chi-square (categorical) and two-tailed T (continous) tests were used to assess the 
differences in characteristics across research groups. Statistical significance levels are indicated 
as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
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Comparison of Survey Respondents with the Fielded Sample and 
the Report Sample: Houston 

This section examines whether impacts among survey respondents in Houston can be 
generalized to the fielded sample and the report sample and whether any cohort effects may 
have been introduced. 8 

Appendix Table F.9 shows the adjusted means and impacts on employment and welfare 
outcomes for Houston’s full sample, fielded sample, and respondent sample. 9 The pattern of im- 
pacts differs across these samples, and so a great deal of caution should be exercised when making 
i nf erences from the survey sample to the report sample in Houston. For example, Appendix Table 
F.9 shows that, among the full report sample, ERA reduced earnings in Year 1 by $94, compared 
with a $228 increase among the fielded survey sample and a $728 increase among the survey re- 
spondent sample. Among the full sample, ERA reduced the percentage working in any given 
quarter by 1.0 percentage point, compared with a 3.7 percentage point increase among the fielded 
sample and a 5.0 percentage point increase among the survey respondent sample. Likewise, the 
ERA and control group differences in the percentage ever employed and food stamp receipt are 
larger among the respondent sample than among the fielded sample and the report sample. The 
fact that these differences are not statistically significant, however, may suggest that random varia- 
tion rather than an actual cohort effect may explain the differences in impacts. 

This potential response bias cannot be rectified by weighting based on background 
characteristics, because differences in measurable characteristics are not systematic. Since the 
overall impacts on administrative records measures are much stronger among those who were 
selected for the survey (and stronger still among survey respondents), it is not surprising that the 
survey impacts look somewhat better in Houston than would be expected from the full sample’s 
impacts on administrative records outcomes. 



Conclusions About Houston 

Due to the narrow cohort of months from which Houston’s survey sample was selected 
— as well as the different pattern of impacts among the survey respondent sample — great cau- 
tion should be exercised when generalizing the results among the survey sample to the broader 
report sample. While the differences in background characteristics are not large, the impacts are 
different across subsamples, which tends to make the survey results appear to be more favorable 
than the full sample’s results. 



8 The impacts presented in this appendix are statistically significant unless otherwise noted. 

9 A11 the impacts are regression-adjusted within each sample, to control for differences in background char- 
acteristics, prior earnings, prior employment, prior public assistance receipt, location or residence, and period 
of sample intake. 
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The Employment Retention and Advancement Project 
Appendix Table F.9 



Comparison of Impacts for the Full, 
Fielded, and Respondent Samples in Houston 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


Ouarters 2-5 

Ever employed (%) 


Full sample 


64.6 


63.8 


0.8 


Fielded sample 


66.8 


61.3 


5.5 


Respondent sample 


67.9 


60.0 


7.9 


Average quarterly employment (%) 


Full sample 


42.8 


43.9 


-1.0 


Fielded sample 


45.3 


41.6 


3.7 


Respondent sample 


46.1 


41.2 


5.0 


Employed 4 consecutive quarters (%) 


Full sample 


21.4 


23.4 


-2.0 


Fielded sample 


24.2 


21.7 


2.5 


Respondent sample 


24.7 


21.1 


3.6 


Number of quarters employed 


Full sample 


1.7 


1.8 


0.0 


Fielded sample 


1.8 


1.7 


0.1 


Respondent sample 


1.8 


1.6 


0.2 


Earnings ($) 


Full sample 


3,851 


3,946 


-94 


Fielded sample 


3,959 


3,730 


228 


Respondent sample 


4,322 


3,594 


728 


Ever received TANF (%) 


Full sample 


85.6 


84.5 


1.1 


Fielded sample 


82.3 


83.1 


-0.8 


Respondent sample 


84.4 


84.0 


0.4 


Amount of food stamps received ($) 


Full sample 


2,766 


2,743 


24 


Fielded sample 


2,824 


3,002 


-178 


Respondent sample 


2,888 


3,125 


-237 


Total measured income ($) 


Full sample 


7,927 


7,897 


30 


Fielded sample 


8,116 


7,984 


132 


Respondent sample 


8,568 


8,003 


564 



SOURCE: MDRC calculations from administrative records from the State of Texas. 



NOTES: The full sample includes 1,950 sample members; experimental: 970; control: 
980. The fielded sample includes 370 sample members; experimental: 187; control: 

183. The respondent sample includes 297 sample members; experimental: 150; control: 
147. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social policy research organization dedicated to learn- 
ing what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness 
of social and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best 
known for mounting rigorous, large-scale, real-world tests of new and existing policies 
and programs. Its projects are a mix of demonstrations (field tests of promising new pro- 
gram approaches) and evaluations of ongoing government and community initiatives. 
MDRC’s staff bring an unusual combination of research and organizational experience to 
their work, providing expertise on the latest in qualitative and quantitative methods and 
on program design, development, implementation, and management. MDRC seeks to 
leam not just whether a program is effective but also how and why the program’s effects 
occur. In addition, it tries to place each project’s findings in the broader context of related 
research — in order to build knowledge about what works across the social and education 
policy fields. MDRC’s findings, lessons, and best practices are proactively shared with a 
broad audience in the policy and practitioner community as well as with the general pub- 
lic and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of pol- 
icy areas and target populations. Once known primarily for evaluations of state welfare- 
to-work programs, today MDRC is also studying public school refomis, employment 
programs for ex-offenders and people with disabilities, and programs to help low-income 
students succeed in college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Child Development 

• Improving Public Education 

• Promoting Successful Transitions to Adulthood 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the 
United Kingdom, MDRC conducts its projects in partnership with national, state, and 
local governments, public school systems, community organizations, and numerous pri- 
vate philanthropies. 




